Intro: Food Environment Atlas¶

The FEA is a publicly available dataset compiled by the US Department of Agriculture. According to the website,

Food environment factors—such as store/restaurant proximity, food prices, food and nutrition assistance programs, and community characteristics—interact to influence food choices and diet quality. These interactions are complex and more research is needed to identify causal relationships and effective policy interventions.

The objectives of the Atlas are:

to assemble statistics on food environment indicators to stimulate research on the determinants of food choices and diet quality, and

to provide a spatial overview of a community's ability to access healthy food and its success in doing so."

This is a compilation of over 275 food environment indicators for every U.S. county. The data were collected from a variety of sources between the years 2007 and 2016.

The objective is to predict the adult obesity rate for 2013 (see the documentation for more details) based on the available data.

Table of Contents¶

1. Miscellaneous Functions
2. Acquiring the Data
   2.1. Consolidating the Data
   2.2. Formatting the Data and Performing Quality Checks
3. Exploratory Analysis
4. Additional Preparations
   4.1. Feature Selection
   4.2. Exploring Collinearity
   
5. Training the Models
   5.1. Baseline Model Performance
   5.2. Hyperparameter Tuning
   5.3. Learning Curves
   5.4. Validation Curves
   
6. Predicting the Obesity Rates
7. Understanding the Results
   7.1. Where the Model Performs Poorly
   7.2. Identifying Feature Importance
   

Misc. Functions¶

Acquiring the Data¶

Create one big data frame¶

Merge the individual tables to make one master dataframe.

Data types are all numbers except for two, STATE and COUNTY.

Add population data. We need to work a little bit on the column names in order to merge this df seamlessly onto the master data frame.

Let's add some relevant columns from the 2013 US Census data since the main dataset only contains poverty rate and median income for 2015.

Drop empty header rows and reset the index. Select only the columns we want to append to the master data frame.

Rename columns, convert to the appropriate data types, and drop rows containing overall state data.

Merge with the main data frame.

Format and check the data¶

Inspect the variable list to figure out which data columns should be dropped. We want to discard data columns that are redundant (e.g., % change given values from two years' worth of data, which are already included as part of the data set) or non-normalized to the population (e.g., counts). Let's also discard data from 2014 and beyond as we'll make predictions for 2013.

Use the selected column names to filter the master data frame. Also drop >2013 columns added from the supplemental table.

Change the data type of categorical variables. Also there's one column that is an integer column, which we can safely change to a float just for uniformity.

Spot check the number of counties attributed to each state. GA has 159 counties compared to CA's 58! Remarkable.

Perform additional basic checks for problematic data.

There are some feature columns (float) that are mostly 0. For instance, there are some counties with no Pacific Islander population and no recreation & fitness facilities. We will drop columns with 10% of the data = 0.

Next, we will examine missing data. Most of the columns that are missing >100 rows seem hyperspecific and understandably irrelevant to many counties, for example FRESHVEG_ACRES07, the number of acres of vegetables harvested for fresh marke in 2007. For simplicity, let's drop the columns with >300 rows of missing data. This will remove mainly agriculture/farm-related columns.

Let's deal with the missing data via imputing. We will fill in missing rows using the median value for each state. For columns that are missing data for entire states, we will enter the national median value.

Exploratory Analysis¶

Let's begin by exploring the distributions of some parameters. First, our target variable: adult obesity rate is the percent of the population with age > 20 and BMI > 30.

Just as an aside, construct a QQ plot and run the Shapiro-Wilk test to test normality ($H_0$=normal distribution). We see that these two tests rule out normality for our obesity rate distribution.

For linear regression: normality in the dependent variable is not a required assumption. Rather, the assumption is that the residual errors—any variation not explained by the features—are normally distributed. Some references say that residual normality does not need to be. satisfied as the sample size increases (CLT)

Let's look at some other parameter distributions. There's a pretty remarkable spread in the population size of the county (also, look how log-symmetric-ish!). Adult diabetes rate is the percentage of the population with age > 20 and diabetes (e.g., types 1 and 2, and excluding gestational diabetes). Poverty rate is the percentage of the population with income below the poverty threshold, which varies by family size, age of the head of the household, etc. For 2017, it was approximately $12,500 for an individual and $25,100 for a family of 4. Low access to store is defined as the percentage of the population living > 1 mile (> 10 miles) from the nearest supermarket or grocery store in an urban (rural) area. The last distribution is strongly skewed to higher % of low access to store.

Let's briefly examine the counties that have 100% of their population experiencing low access to store to understand the extreme values. After spot-checking a few counties along with a cross-reference Google map search, it seems that what appear to be regular grocery stores do exist near residential areas in these counties. Some guesses for why these counties are labeled to have low access to grocery stores for ~100% of their population are 1. these grocery stores do not meet the somewhat strict definition of "supermarket/large grocery store," e.g., more than $2 million in annual sales, 2. population data were not properly allocated to their 0.5 sq. km grids. The 100% designation does appear to be correct for some of these counties (e.g., for Kiowa, CO, the nearest large grocery store, Safeway, is ~8 miles from town center) while for others, it is more ambiguous. Thus we will proceed without attempting to correct or remove these potential outliers, especially given their small numbers.

We can also examine the prevalence of adult obesity broken down separately for metro/nonmetro counties and counties with persistent poverty (poverty rate > 20% in the 1980, 1990, and 2000 censuses and the American Community Survey 5-year estimates for 2007-2011).

Let's now move onto more explicitly examining relationships between obesity and the available parameters.

The positive trend between poverty and obesity rate is not too surprising if we are to believe that poverty is associated with poor food choices, which is in turn associated with poor health/obesity. There is a large spread in the obesity rate values as a function of average milk price, though there may be a weak non-linear relationship. There isn't a super obvious, strong correlation between the prevalence of FF restaurants and obesity rates. Based on studies such as this, we know that obesity disproportionately affects African-Americans and Mexican-Americans. This is confirmed by the data (for the African-American population at least, as separate Mexican-American data is not available).

We can examine the same distributions broken up by race (above/below the national average value according to the 2010 US. Census). It's interesting that obesity and diabetes rate distributions for counties with the fraction of the black/African-American population exceeding vs. below the national average (12.6%) show visible differences. This does not hold true for counties with fraction of the white population exceeding/below the national average (72.4%).

Next, although it may seem totally trivial, let's inspect poverty rate vs. median household income. As expected, there's a negative trend, but the exact form of this scatterplot is pretty interesting. The steep rise below the poverty rate of 10% implies that even in low-poverty areas, there's actually a large variability in the overall affluence of the population (ranging from ~40k to over 100k in median household income). Similarly, the long tail in the poverty rate beyond 10% implies that there are issues beyond the median income at play (cost of living is one example). At a fixed poverty rate, counties with higher % of white population seems to have higher median household incomes.

Just for fun, let's select a random subset of the counties and inspect the breakdown in race.

Get a list of features that have outliers as determined by the IQR score for reference later. Outliers are understandable and even expected for certain features like ethnicity and population.

Additional Prepwork¶

We have to do a few more things before we can throw the data at ML algorithms.

We will rename the data to X following the general convention, then split into training and test sets. We'll remove 'PCT_DIABETES_ADULTS' columns because Type II diabetes is strongly associated with obesity ("People who are obese -- more than 20% over their ideal body weight for their height -- are at particularly high risk of developing type 2 diabetes and its related medical problems.") and we are interested in obesity/diabetes as predictions of food environment factors.

The features that will be used to predict the obesity rate are listed below. Many of these features appear to be various representations/encodings of poverty/socioeconomic status.

Some features are better log'd.

Next, standardize the features so they are all on the same order of magnitude. Apply the same transformation to the test set. Let's proceed without the binary features 1. for simplicity and 2. the features don't show sensitivity to our target feature.

Dealing with Collinearity¶

We will remove feature columns with VIF > 10.

Note: Collinearity can result in unstable coefficients for linear regression. Although this is not a big concern for decision trees but feature importance can be harder to interpret.

It's interesting to observe the correlation matrix as well.

Training ML algorithms¶

Establishing a Baseline¶

Before we proceed, let's first establish a baseline performance to compare with subsequent results. A very simple baseline guess is the median of the target values. We will use two metrics, MAE (mean absolute error, so we want to minimize) and adjusted $R^2$ (MSE/variance in $y$ weighed by the numbers of observations and features, so we want to maximize to 1).

Unlike $R^2$, adjusted $R^2$ will not arbitrary increase with an increasing number of terms if the added complexity in the model does not actually improve the prediction.

Another popular choice is RMSE (root mean square error, i.e., standard deviation) because it's easily and smoothly differentiable, but the squaring makes it pretty sensitive to outliers.

We will evaluate the generalized performance on the following ML algorithms, with default choices of the hyperparameters for now.

• Linear Regression (without regularization, L1/L2 regularizations: features vs y are non-linear, so probably not a great choice here)
• K Nearest Neighbors
• Gradient Boosting (we will use the sklearn implementation)
• Random Forest
• Suppert Vector Machine

Hyperparameter Tuning¶

RF and GBR both seem promising. RF is nicer to interpret, and it gives feature importance, so let's go with it.

Rather than searching the hyperparameter space via grid search (brute-force) or randomized search (slight improvement over grid search), we will turn to a Bayesian optimization technique (tree parzen estimator) via hyperopt.

First define an objective function that returns a loss to be minimized. We will use a cross validation score so that we don't have to split the data further into training & validation sets.

Define the characteristics of the hyperparameter space over which we will conduct our search. We will choose uniform distributions for simplicity.

Store the training information.

Hyperopt returns the parameters as floats, so convert the relevant ones back to integer data types.

Learning Curve¶

Before we move on to deploying the best model, let's construct a learning curve to examinie the bias-variance trade-off.

As expected, the test (CV) error is larger than the training error (CV error was computed on previously unseen data). There appears to be some overfitting (difference in MAE of ~1, or error in the obesity rate of 1%), meaning the current model has high variance. Getting more data is not really an option, since we are limited to the # of counties in the U.S. Let's move on to examining the validation curves below and see how individual parameters of our model influence this bias-variance trade off.

Validation Curves¶

We will also plot the validation curve for a few hyperparameters.

First, n_estimators is the number of decision trees. We see that the two errors quickly diverge beyond ~50, and CV errors asymptotes while training error continues to decrease: as we increase the number of estimators, the training set becomes increasingly overfit.

The deeper the individual tree goes, the more it will overfit.

The fraction of features used in each split in the decision tree.

Note that all CV errors plateau to ~2% regardless of the choice of parameters, # samples, etc.

Predict the Obesity Rates¶

Train on the full train data set then check the MAE and the adjusted $R^2$ scores on the test set.

In terms of mean absolute error, this is a nearly 70% improvement over the naive baseline model (median)!

Feature Importance¶

Let's examine which features played a crucial role in predicting the obesity rates. The feature importance of a single predictor variable is defined by combining the following: the fraction of the samples split according to the said variable—a measure of its "sphere of influence," if you will—and the decrease in Gini impurity resulting from the said split. Feature importances are computed for a given decision tree, and these values are averaged over the ensemble of trees to provide the final list of feature importances that we will examine here.

The most importance predictors by far are related to socioeconomic status.