ESTIMATING THE HETEROGENEOUS WELFARE EFFECTS OF CHOICE ARCHITECTURE

• Date: 01 August 2019
• DOI: http://doi.org/10.1111/iere.12382
• Published date: 01 August 2019

INTERNATIONAL ECONOMIC REVIEW

Vol. 60, No. 3, August 2019 DOI: 10.1111/iere.12382

ESTIMATING THE HETEROGENEOUS WELFARE EFFECTS OF CHOICE

ARCHITECTURE∗

BYJONATHAN D. KETCHAM,NICOLAI V. KUMINOFF,AND CHRISTOPHER A. POWERS1

Arizona State University, U.S.A.; Arizona State University, U.S.A., and National Bureau of

Economic Research, U.S.A.; Cigna-HealthSpring, U.S.A.

We develop a method that embeds signals about consumers’ knowledge to evaluate prospective choice

architecturepolicies. We analyze three proposals for U.S. Medicare prescription drug insurance markets: (i) menu

restrictions, (ii) personalized information, and (iii) defaulting consumers to cheap plans. We link administrative

and survey data to identify informed enrollment decisions that proxy for preferences of observationally similar

misinformed consumers. Results suggest that each policy yields winners and losers, with the menu restrictions

harmful to most but personalized information beneficial to most. These results are robust across signals of

consumers’ knowledge but differ from the benchmark that excludes such signals.

1. INTRODUCTION

One of the frontiers in empirical microeconomics is to assess the equity and efficiency of

polices that alter a market’s design and “nudge” consumers toward making certain decisions.

Thaler and Sunstein (2008) denote this approach to policy as “choice architecture.” Examples

of choice architecture include restricting the number of differentiated products in a market,

providing consumers with personalized information about their options, and making default

choices for consumers but letting them opt out. Numerous government organizations including

the United States and the World Bank have begun using choice architecture to nudge the

beneficiaries of public programs.

A stated goal of choice architecture is to benefit consumers who do not make fully informed

decisions. Such paternalistic policies may also harm some consumers by eliminating their pre-

ferred products, by making it harder to buy those products, and by causing prices to increase.

Despite the potential for important and heterogeneous effects, little work has predicted the

distribution of gains and losses of prospective choice architecture policies. To do so within a

revealed-preference framework requires addressing two challenges. First, analysts must identify

which decisions are misinformed and hence potentially misleading about consumer preferences.

Second, analysts must infer the preferences of both informed and misinformed consumers. In

∗Manuscript received October 2017; revised September 2018.

1Ketcham and Kuminoff’s research was supported by a grant from the National Institute for Health Care Man-

agement (NIHCM) Research and Educational Foundation. The findings do not necessarily represent the views of the

NIHCM Research and Education Foundation. The views expressed in this article are those of the authors and no official

endorsement by the Department of Health and Human Services or the Centers for Medicare & Medicaid Services is

intended or should be inferred. We are grateful for insights and suggestions from Gautam Gowrisankaran, Kate Ho,

S´

ebastien Houde, Mike Keane, Don Kenkel, Christos Makridis, Alvin Murphy, Sean Nicholson, Jaren Pope, Dan Sil-

verman, Meghan Skira, V. Kerry Smith, and seminar audiences at the AEA/ASSA Annual Meeting, the Congressional

Budget Office, Health and Human Services Office of the Assistant Secretary for Planning and Evaluation, the ASU

Health Economics Conference, the Annual Health Economics Conference, the Quantitative Marketing and Economics

Conference, the Health Econometrics Conference, Boston College, Brigham Young University, Cornell University,

Iowa State University, Michigan State University, Northern Arizona University, Stanford University, University of

Arizona, UC Santa Barbara, University of Calgary, University of Chicago, University of Maryland, University of Mi-

ami, University of Southern California, Vanderbilt University, and Yale University. Please address correspondence to:

Jonathan D. Ketcham, Department of Marketing and Department of Economics, Arizona State University, Tempe,

AZ 85287. E-mail: Ketcham@asu.edu

1171

C

(2019) by the Economics Department of the University of Pennsylvania and the Osaka University Institute of Social

and Economic Research Association

1172 KETCHAM,KUMINOFF,AND POWERS

this article, we develop an empirical framework to address both challenges and use it to eval-

uate policies that have been proposed to nudge consumer decision making in health insurance

markets.

We operationalize Bernheim and Rangel’s (2009) conceptual logic for policy analysis in the

presence of latent constraints that undermine revealed preference logic for some consumers. We

envision consumers facing differentiated costs of acquiring information about their choice sets,

so that some consumers may choose to purchase products without becoming fully informed.

Analysts often observe the characteristics of consumers, their choices, and their choice sets

but typically do not observe how consumers form beliefs or make decisions. In our context, we

observe signals about whether each decision was made by an informed consumer. First, we have

access to the results of survey-based tests of consumers’ knowledge about the products they are

choosing. Second, we observe each person’s full menu of choices, their actual choice outcomes,

and the counterfactual outcomes under each option available to them. With this information,

we develop signals of whether the choices reveal or conceal preferences, such as whether their

survey responses indicate comprehension of key market institutions and whether their choices

are consistent with axioms of consumer theory. We examine how these signals correspond to

proxies for being informed, such as the presence of Alzheimer’s disease, educational levels, and

self-reported efforts to gather information. We use these signals to identify the subset of choices

that we suspect may fail to reveal preferences. We show that welfare analysis is possible in this

setting if the mapping between preferences and consumer demographics is stable across the

groups of consumers making “suspect” and “nonsuspect” choices.2Under this stability assump-

tion, we estimate a repeated choice multinomial logit model that incorporates heterogeneity on

observed consumer attributes and we derive welfare measures that characterize how hetero-

geneous consumers are affected by choice architecture policies. Our measures are consistent

with the idea that consumer inertia may arise from a latent mixture of preferences, information

costs, switching costs, and psychological biases. In the special case where all consumers are fully

informed, freely mobile and immune to biases, our welfare measures reduce to those derived

by Small and Rosen (1981).

We use our model to study financial decisions among elderly Medicare beneficiaries in the

United States. The elderly population is particularly important because they control a large

share of wealth and frequently experience declines in cognitive function (Fang et al., 2008;

Agarwal et al., 2009; Querfurth and LaFerla, 2010; Keane and Thorp, 2016). We analyze their

choices in markets for Medicare stand-alone prescription drug insurance plans (PDPs). In 2015,

these government-designed, taxpayer-subsidized markets annually enrolled 25 million older

adults with federal outlays of $75 billion (U.S. Department of Health and Human Services,

2017). Beneficiaries’ enrollment decisions are multifaceted and financially important. Between

2006 and 2010, the average new enrollee chose among 50 plans that differed in cost, risk

protection, and quality. Returning enrollees were automatically reassigned to their previously

chosen plans unless they opted to switch plans during the annual open enrollment window. The

median enrollee spent approximately 6% of her annual household income on premiums and

out-of-pocket (OOP) costs.

Due to concerns about expenditure levels, market complexity, and consumer inertia, re-

searchers and federal agencies have proposed several reforms to prescription drug insurance

markets (McFadden, 2006; Thaler and Sunstein, 2008; Federal Register, 2014). These include

reducing the number of plans, providing consumers with personalized information about their

options, and auto-assigning people to default plans that are expected to minimize cost. We

assess the welfare effects of these proposals by combining administrative records and survey

data on a national panel of enrollees from 2006 to 2010. Specifically, we link the longitudinal

2The “suspect/nonsuspect” terminology is borrowed from Bernheim and Rangel (2009). This language emphasizes

that it is virtually impossible for analysts to determine beyond a doubt whether a consumer is fully informed at the time

of her decision. Our framework requires only that nonsuspect choices be fully informed. Suspect choices may or may

not be fully informed.

WELFARE EFFECTS OF CHOICE ARCHITECTURE 1173

Medicare Current Beneficiary Survey (MCBS) to administrative records of the respondents’

annual enrollment decisions, drug claims, and medical conditions. This novel linkage allows us

to combine information on enrollees’ efforts to learn about the market, their knowledge of how

products differ, whether they self-enrolled in plans or had help from advisors, their demograph-

ics, their choices and their choice outcomes, and their health, including their prescription drug

utilization.

We capitalize on the depth and breadth of these linked data to develop several signals

of consumers’ knowledge given that any one signal is potentially controversial. Our primary

approach is to assume that a decision is informed if two conditions are satisfied: (i) the decision

maker’s performance on the MCBS knowledge test demonstrates that she understands that her

OOP prescription drug costs vary across plans and (ii) the plan choice can be rationalized by a

preference ordering that is complete, transitive, monotonic, and weakly risk averse. These two

requirements are jointly satisfied for 58% of enrollment choices. Enrollees in this nonsuspect

group tend to be better educated and have exerted more effect to learn about the market.

They also tend to be younger, to have fewer drug claims, and are less likely to be diagnosed

with Alzheimer’s disease or other forms of dementia. Our secondary approaches to partitioning

decisions include using the knowledge test alone, using other choice outcomes based on either

ex ante or ex post drug consumption, and using other combinations of the two. This variety

of measures also allows us to provide new insights about what specific knowledge consumers

may be lacking. Although the measures based on choice outcomes incorporate consumers’

knowledge about plans as well as their individual-specific drug needs, the measure based on the

MCBS question isolates consumers’ knowledge about plan design specifically.

After dividing choices into suspect and nonsuspect groups, we estimate and validate multi-

nomial logit models for each group, incorporating heterogeneity within each group in terms of

income, education, race, age, sex, prescription drug use, and information-seeking efforts. We

model annual plan choices as a static repeated-choice process with a cost of switching plans.3

The results show that enrollees in the nonsuspect group are sensitive to price and risk averse at

levels consistent with evidence from other insurance markets (Cohen and Einav, 2007; Handel,

2013; Handel and Kolstad, 2015). In contrast, enrollees in the suspect group make choices that

seemingly imply they are risk loving, less price sensitive, and highly averse to switching plans.

We use our estimates to simulate three prospective choice architecture policies. The first

policy is the government’s proposal to limit each insurer to sell no more than two plans per

market (Federal Register, 2014). Second, we calibrate our model to replicate a field experiment

by Kling et al. (2012; henceforth KMSVW) in which enrollees were told which plan would be

cheapest for them and how much money they could expect to save by switching. In the third

experiment, we simulate the government’s proposal to reassign people to their cost-minimizing

plans (Health and Human Services, 2014). Our framework formalizes ways in which each

policy may create winners and losers.4We simulate each policy under a range of assumptions

about consumer foresight, about the causes of inertia, and about how the policies will affect

consumers’ decisions. Specifically, we report the share of consumers who benefit from each

policy and measures of consumer surplus as bounds on ranges that we obtain by repeating our

analyses under the extreme assumptions about the efficacy of choice architecture. In our “most

effective” (ME) scenario, we assume that each policy causes consumers in the suspect group

to behave like their analogs in the nonsuspect group. This scenario also assumes that inertia is

caused entirely by misinformation. At the opposite extreme, our “least effective” (LE) scenario

3A static model is appropriate here because it is difficult for consumers to forecast their own future prescription drug

needs, let alone the drug needs and enrollment decisions of other consumers together with the implications for plan

prices and offerings. Our static approach is similar to other health insurance applications such as Handel (2013) and

Handel and Kolstad (2015).

4Forexample, the menu restrictions may benefit misinformed consumers by reducing their ability to choose low-utility

plans. The information treatment and default assignment policies could create losers due to asymmetric information

because the government would only use prior drug claims and by creating incentives for consumers to choose plans

that are cheaper but potentially lower utility due to lower quality or risk protection.