GENDER BIAS IN OPINION AGGREGATION

• Published date: 01 August 2021
• Author: Friederike Mengel
• Date: 01 August 2021
• DOI: http://doi.org/10.1111/iere.12503

INTERNATIONALECONOMIC REVIEW

Vol. 62, No. 3, August 2021DOI: 10.1111/iere.12503

GENDER BIAS IN OPINION AGGREGATION∗

By Friederike Mengel

University of Essex, U.K. ; Lund University, Sweden

Gender biases have been documented in many areas including hiring, promotion, or performance

evaluations. Many of these decisions are made by committees. We experimentally investigate whether commit-

tee deliberation contributes to gender biases. In our experiments, participants perform a real effort task and

then rate the task performance of other participants. Across treatments we vary the extent of deliberation pos-

sible. We f‌ind that deliberation increases gender biases. We explore several mechanisms and test two interven-

tions. Randomizing the order of speaking does not reduce gender bias, but an information intervention where

raters are informed of gender bias in prior sessions does.

1. introduction

Persistent gender earnings and promotion gaps have attracted much attention in research

and policy debates in recent years; see Goldin and Rouse (2000), Black et al. (2008), or Sand-

berg (2013) among many others.1Indeed, a large body of empirical evidence has documented

gender biases in decisions, such as hiring (Neumark et al., 1996; Goldin and Rouse, 2000), pro-

motion (Booth et al., 2003; Ginther and Khan, 2004; Bagues and Esteve-Volart, 2010; Bagues

et al., 2017), the allocation of venture capital investments (Malmstrom et al., 2018), or perfor-

mance evaluations (Bohnet et al., 2016; Sandberg, 2018). One thing that is common to all of

these decisions is that they involve deliberation by committee members.

In this article, we aim to understand whether committee deliberation contributes to gen-

der biases. Doing so requires (i) measuring pre- and postcommunication beliefs and (ii) an ex-

perimental variation of the amount of deliberation allowed. Conducting a lab experiment en-

ables us to create such a design. In all treatments of our experiment, participants perform a

real effort task, where performance evaluation is subjective. They then rate the performance

of nine other (anonymous) participants. Our treatments systematically vary two features that

distinguish committee decision making from individual decisions: (i) the amount of delibera-

tion possible and (ii) the fact that there are incentives to reach an agreement in the commit-

tee.

∗Manuscript received April 2020; revised September 2020.

I thank Sonia Bhalotra, Steven Bosworth, Irma Clots Figueras, Chiara Franzoni, Ilyana Kuziemko, Michael Naef,

Wieland Mueller, Johanna Rickne, Daniel Sgroi, Camille Terrier, and seminar participants in Amsterdam, Birming-

ham, Essex, Innsbruck, King’s College London (Political Economy workshop), Konstanz, LMU Munich, Lund (Mi-

croseminar and Arne Rhyde Conference on Learning and Evolution), Nice (ASFEE 2018), Nuremberg (Conference

on Gender and the Labour Market 2018), NYU Abu Dhabi, Reading (Behavioural Econ Workshop), and Toulouse

for helpful comments and Sara Godoy, Mihail Morosan, Rafael Brancu, and Flavio Lomaski Torrez for excellent re-

search assistance. Financial support by the European Research Council (ERC Starting Grant 805017-DYNNET) is

gratefully acknowledged. Please address correspondence to: Friederike Mengel, Department of Economics, Univer-

sity of Essex, Wivenhoe Park, Colchester CO4 3SQ, U.K.; Department of Economics, Lund University, SE-220 07

Lund, Sweden. E-mail: fr.mengel@gmail.com

1Academia is no exception. Although in some f‌ields of academia gender promotion and earnings gaps have con-

verged, this is not true in others. Economics is one of the f‌ields where promotion and earnings gaps are particularly

persistent and cannot be easily explained. According to Ceci et al. (2014), earnings gaps among U.S.full professors in

Economics are even higher in 2010 than they were in 1995 with female full professors earning less than 75% of their

male counterparts.

1055

© (2021) by the Economics Department of the University of Pennsylvania and the Osaka University Institute of So-

cial and Economic Research Association

1056 mengel

Our main treatments involve open committee deliberation. Participants f‌irst rate task

performance by other experimental participants. They can then deliberate via a chat window

for three minutes with two other committee members. After the chat, they see the ratings of

the other committee members and submit a revised rating. Committees are given an incentive

payment if all three revised ratings agree.

We then systematically shut down different aspects of committee deliberation. First, we re-

move the chat and with it the possibility to persuade or convince others. Next, we remove the

possibility to exchange information (to see others’ ratings), and last we also remove the incen-

tive payment, leading us to a situation of individual decisions where participants do not have

to think about what others might do. All treatments exist in a variation where gender identity

is revealed (Gvariation) and one where it is not. Comparing the gender-blind and nonblind

treatments identif‌ies the gender bias in the different conditions.

We f‌ind strong and highly statistically signif‌icant gender biases under open committee de-

liberation. After deliberation, 60% of ratings received by men are revised upward compared

to only 25% of ratings received by women. As a consequence women are ranked on average

three positions lower after deliberation.2Shutting down open deliberation and information

exchange removes the gender bias at least as long as there are still incentives to agree. How-

ever, doing so might not be desirable viewed from other perspectives. It might not lead to op-

timal decisions, for example, if decision makers hold different information about candidates. It

might also not be feasible for legal reasons or when communication among committee mem-

bers cannot practically be prevented.

We hence tested two further interventions both designed to reduce gender bias in the pres-

ence of open deliberation. The f‌irst intervention randomized the order of speaking in the

committee. This intervention was unsuccessful and in fact produced weakly larger gender

biases compared to our baseline open-deliberation treatment. The second intervention we

tested is an information intervention, where participants are made aware of gender bias in

previous sessions prior to entering their ratings. Similar interventions have sometimes been

shown to be successful in noncommittee decision making (Boring and Philippe, 2017; Pope

et al., 2018). We also f‌ind that this intervention is successful. There is no gender bias in

this treatment.

These results carry potentially actionable policy consequences. Our interventions have

shown that care must be taken when designing rules for committee deliberation. Changes de-

signed to reduce bias, such as randomizing the order of speaking in a committee, can have un-

intended consequences and in our case led to very strong gender bias. On the other hand, our

information intervention was successful and did not lead to gender bias (neither against men

nor women). We also did not f‌ind evidence that this intervention would lead to greater polar-

ization of opinions.

Our results contribute to two strands of literature that we will review in detail below

(Section 2): (i) literature on information aggregation in groups and (ii) literature on gender

bias. With respect to the former our results point to the importance of institutional detail

when “truth” is subjective. Under minimal communication there is no gender bias and com-

mittees arguably reach a more objective judgement. Under open communication, by contrast,

the group is more biased than the sum of individual ratings would suggest. With respect to

literature on gender bias, we highlight the importance of studying the role of committee de-

cision making in many of the areas where gender biases have been identif‌ied. To our knowl-

edge, our article is the f‌irst to identify the role of committee deliberation for gender biases.

The article is organized as follows. Section 2 discusses literature on information aggrega-

tion and gender biases in more detail and points out how our article contributes to each. Sec-

tion 3 describes the experimental design and procedures. Section 4 contains the main results.

Section 5 discusses the results from two different “interventions” designed to reduce gender

2We also conduct a sentiment analysis (Thelwall et al., 2010), which reveals that chats contain more positive

statements when a summary written by a man is rated compared to when a woman’s summary is rated.