,Judge Business School, University of Cambridge, UK.
Random Forest and Causal Effect Estimation
Jinhua Wang and Melvyn Weeks
In the mid-1970s, a job training program (National Supported Work Demonstration Program) was carried out in the U.S., where the eligible applicants were randomly assigned to treatment and control groups. The candidates in the treatment group received NSW training while the control group did not. Following the random experiment, LaLonde (1986) and Dehejia and Wahba (1999) expanded the NSW random experiment data with non-experiment population samples drawn from the Panel Study of Income Dynamics (PSID) and Current Population Survey (CPS). They then ran a horse race of different parametric econometrics models on the non-experimental data to obtain the treatment effects. By comparing the treatment effects obtained with random experiment data with the treatment effect obtained in non-random experiment data, LaLonde (1986) and Dehejia and Wahba (1999) show that not all parametric model, such as propensity score matching, provide reliable results when applied to observational (i.e. non-random experiment) data. We attack the same problem with Random Forest, which is one of the Machine Learning methods that are fast to train and less prone to overfitting (Breiman 2001). The fundamental question is to find out the counterfactual – what would have happened if the person in the treatment group were in the control group. Observing the counterfactual is almost impossible, and the ideal thing to do is to have a random experiment so that on average, the treatment effect is simply the difference between the treatment and control group samples. However, it is not always possible to have a completely random experiment. In the NSW sample, for example, the randomisation was done over a three-year period and the population that joined earlier has some difference in characteristics from the population that joined later. Other than the self-selection effect, the introduction of the PSID and CPS sample also introduces another layer of confoundedness. To find out the average treatment effect under the possible influence of confoundedness, the best we can do is to identify individuals with similar characteristics and calculate the treatment effects between the treated individuals and the untreated, but similar, individuals. Parametric methods, such as propensity score matching, asserts strong parametric assumption on the model, and it only provides a single index for matching samples with similar characteristics. Random Forest, on the other hand, relaxes the parametric assumption by training an ensemble of decision trees while minimising the classification error. Over the past years, Random Forest had excelled at making classifications and predictions. We show that Random Forest can also provide great estimations of treatment effects by classifying individuals with similar characteristics into the same leaf of a decision tree. We use several different variants of Random Forest to estimate the treatment effects of the NSW dataset, including Causal Tree, Double Sample Causal Forest, Propensity Causal Forest, and Propensity Matching with Probability Forest. In particular, we adopt the Athey and Imbens (2015) methodology called Honest Estimation. We also explore the heterogeneity of treatment effects with Causal Trees.
Jinhua is a PhD student at Judge Business School, University of Cambridge. He is currently a member of the Finance subject group and has active research interests in Machine Learning and Economics