2001 Lawrence R. Klein Lecture Estimating Distributions of Treatment Effects with an Application to the Returns to Schooling and Measurement of the Effects of Uncertainty on College Choice*

• Author: James J. Heckman,Karsten T. Hansen,Pedro Carneiro
• DOI: http://doi.org/10.1111/1468-2354.t01-1-00074
• Date: 01 May 2003
• Published date: 01 May 2003

INTERNATIONAL

ECONOMIC

REVIEW

May 2003

Vol. 44, No. 2

2001 LAWRENCE R. KLEIN LECTURE

ESTIMATING DISTRIBUTIONS OF TREATMENT EFFECTS

WITH AN APPLICATION TO THE RETURNS TO SCHOOLING

AND MEASUREMENT OF THE EFFECTS OF UNCERTAINTY

ON COLLEGE CHOICE∗

BYPEDRO CARNEIRO,KARSTEN T. HANSEN,AND JAMES J. HECKMAN1

Department of Economics, University of Chicago; Kellogg School

of Management, Northwestern University; Department of Economics,

University of Chicago and The American Bar Foundation

This article uses factor models to identify and estimate the distributions of

counterfactuals. Weextend LISREL frameworks to a dynamic treatment effect

setting, extending matching to account for unobserved conditioning variables.

Using these models, we can identify all pairwise and joint treatment effects. We

apply these methods to a model of schooling and determine the intrinsic uncer-

tainty facing agents at the time they make their decisions about enrollment in

school. We go beyond the “Veil of Ignorance” in evaluating educational policies

and determine who benefits and who loses from commonly proposed educational

reforms.

∗Manuscript received October 2000; revised January 2003.

1Previous versions of this paper were given at the Midwest Econometrics Group,Chicago, October

2000, Washington University St. Louis,May, 2001, the Nordic Econometrics Meetings, May,2001 and

workshops at Chicago August, 2002 and Stanford, January, 2003. A simple version of this paper is

presented in Carneiro, Hansen, and Heckman (2001). A version of this paper was presented by Heck-

man as the Klein Lecture at the University of Pennsylvania, September, 28, 2001 and also at the IFAU

conference in Stockholm Sweden, October 2001. Weare grateful to all workshop participants. Weespe-

cially thank Mark Duggan, Orazio Attanasio,and Michael Keane for comments on the first draft of this

paper. We have benefited from discussions with Ricardo Barros,Richard Blundell, Francisco Buera,

Flavio Cunha, Mark Duggan, Lars Hansen, Steven Levitt, Bin Li, Luigi Pistaferri, and Sergio Urzua

on subsequent drafts.We single out Salvador Navarro and Edward Vytlacil for especially helpful com-

ments. Weare grateful to Flavio Cunha and Salvador Navarro for exceptional research assistance and

hard work. This research is supported by NSF 97-09-873, SES-0099195, and NICHD-5RO1-HD34958.

Heckman’s work was also supported by the American Bar Foundation and the Donner Foundation,

Pedro Carneiro’sresearch was supported by Funda ¸c˜ao Ciˆenciaand Tecnologiaand Funda ¸c˜ao Calouste

Gulbenkian. Please address correspondence to: JamesJ. Heckman, Department of Economics,Univer-

sity of Chicago, 1126 E. 59th Street, Chicago,IL 60637, USA. Tel: +773 702-0634. Fax: +773 702-8490.

E-mail: jjh@uchicago.edu.

361

362 CARNEIRO,HANSEN,AND HECKMAN

1. INTRODUCTION

The recent literature on evaluating social programs finds that persons (or firms

or institutions) respond to the same policy differently (Heckman, 2001). Thedistri-

bution of responses is usually summarized by some mean. A variety of means can

be defined depending on the conditioning variables used. Different means answer

different policy questions. There is no uniquely defined “effect”of a policy.

The research reported here moves beyond means as descriptions of policy out-

comes and determines joint counterfactual distributions of outcomes for alterna-

tive interventions. Fromthe knowledge of the joint distributions of counterfactual

outcomes it is possible to determine the proportion of people who benefit or lose

from making a particular policy choice (taking or not taking particular treatments),

the origin and destination outcomes of those who change states because of policy

interventions, and the amount of gain (or loss) from various policy choices by

persons at different deciles of an initial prepolicy distribution. Our work builds

on previous research by Heckman and Smith (1993, 1998) and Heckman et al.

(1997) that uses experimental data to bound or point-identify joint counterfac-

tual distributions. We extend the analysis of Aakvik et al. (1999, 2003), who use

factor models to identify counterfactual distributions to consider indicators for

unobservables, implications from choice theory, and to exploit the benefits of

panel data.

From the joint distribution of counterfactuals,it is possible to generate all mean,

median, or other quantile gains, to identify all pairwise treatment effects in a

multi-outcome setting, and to determine how much of the variability in returns

across persons comes from variability in the distributions of the outcome selected

and how much comes from variability in opportunity distributions. Using the

joint distribution of counterfactuals, it is possible to develop a more nuanced

understanding of the distributional impacts of public policies,and to move beyond

comparisons of aggregate overall distributions induced by different policies to

consider how people in different portions of an initial distribution are affected

by public policy. We extend the analysis of DiNardo et al. (1996) to consider

self-selection as a determinant of aggregate wage and earnings distributions.

Using our methods, we reanalyze the model of Willis and Rosen (1979), who

apply the Roy model (1951) to the economics of education. Weextend their model

to account for uncertainty in the returns to education. Wealso distinguish between

present value income-maximizing and utility-maximizing evaluations of schooling

choices and we estimate the net nonpecuniary benefit of attending college. We

use information on the choices of agents to determine how much of the ex post

heterogeneity in the return to schooling is forecastable at the time agents make

their schooling choices. This procedure extends the analysis of Flavin (1981) to

a discrete choice setting. This allows us to identify the effect of uncertainty on

schooling choices.Ex ante, there is a great deal of uncertainty regarding the returns

to schooling (in utils or dollars). Ex post, 8% of college graduates regret going to

college.

The plan of this article is as follows. Section 2 presents the essential idea un-

derlying the identification strategy used in this article and how our approach is

EFFECTS OF UNCERTAINTY ON COLLEGE CHOICE 363

related to previous work. Section 3 presents a general policy evaluation frame-

work for counterfactual distributions with multiple treatments followed over time.

The strategy pursued in this article is based on using low-dimensional factors to

generate distributions of potential outcomes. We show how our methods general-

ize the method of matching by allowing some or all of the variables that generate

the conditional independence assumed in matching to be unobserved by the ana-

lyst. Section 4 introduces the factor models used in this article. Section 5 presents

proofs of semiparametric identification. Section 6 applies the analysis to extend the

Rosen–Willis model of college choice to account for uncertainty and to estimate

the information about future earnings available to agents at the time schooling

decisions are made. Section 7 reports estimates of the distributions of returns to

schooling, the components unforecastable by the agent at the time schooling deci-

sionsaremade,and the nonpecuniary net benefits from attending college. Section 8

applies our estimates to evaluate a reform of the U.S. educational system. It illus-

trates the power of our method to lift the commonly invoked veil of ignorance

and move beyond aggregate distributions of outcomes to understand the conse-

quences of public policies on persons in various parts of the overall distribution.

Section 9 concludes. We first provide a brief introduction to the literature to put

this article in context.

2. ESTIMATING DISTRIBUTIONS OF COUNTERFACTUAL OUTCOMES

In order to place the approach used in this article in the context of an emerging

literature on heterogeneous treatment effects,it is helpful to motivate our work by

a two-outcome, two-treatment cross section model. Forsimplicity, in this section it

is assumed that the outcomes are continuous random variables. Theanalysis in the

rest of this article is for multiple treatments and multiple outcomes followed over

time, and the outcomes may be discrete,continuous, or mixed discrete-continuous.

The agent can experience one of two possible counterfactual states with asso-

ciated outcomes (Y

0,Y

1). The states are schooling levels in our empirical analysis.

Xis a determinant of the counterfactual outcomes (Y

0,Y

1); S=1 if the agent is

in state 1; S=0 otherwise. The observed outcome is Y=SY

1+(1 −S)Y

0. There

may be an instrument (or set of instruments) Zsuch that (Y

0,Y

1)⊥⊥ Z|Xand

Pr(S=1|Z,X) depends on Zfor all X(so it is a nontrivial function of Z), i.e., Z

is in the choice probability but not the outcome equation. (A⊥⊥ B|Cmeans Ais

independent of Bgiven C). We show below that such a Zis not strictly required in

our approach. The standard treatment effect model assumes policies (Z) that af-

fect choices of treatment but not potential outcomes (Y

0,Y

1). General equilibrium

effects are ignored.2

Thegoal of our analysis is to recover F(Y

0,Y

1|X). As noted in Heckman (1992),

Heckman and Smith (1993, 1998), and Heckman et al. (1997), from this joint distri-

bution it is possible to estimate the proportion of people who benefit (in terms of

gross gains) from participation in the program (Pr(Y

1>Y

0|X)), gains to partici-

pants at selected levels of the no-treatment distribution (F(Y

1−Y

0|Y

0=y0,X)),

2See Heckman et al. (1998a, 1998b, 1998c, 2000) for a treatment of general equilibrium policy

evaluation.