THE FACE VALUE OF ARGUMENTS WITH AND WITHOUT MANIPULATION

• DOI: http://doi.org/10.1111/iere.12479
• Author: Fangya Xu,Mike Felgenhauer
• Published date: 01 February 2021
• Date: 01 February 2021

INTERNATIONALECONOMIC REVIEW

Vol. 62, No. 1, February 2021 DOI: 10.1111/iere.12479

THE FACE VALUE OF ARGUMENTS WITH AND WITHOUT MANIPULATION∗

By Mike Felgenhauer and Fangya Xu1

Brunel University London, UK; University of Reading, UK

A sender wishes to persuade a receiver with a (surprising) result that challenges the prior belief. The result

stems either from sequential private experimentation or manipulation. The incentive to experiment and to ma-

nipulate depends on the quality threshold for persuasion. Higher thresholds make it harder to f‌ind a surprising

outcome via experimentation and may encourage manipulation. Suppose there are observable nonmanipula-

ble and manipulable research methods. For the decision quality, the quality threshold for persuasion for non-

manipulable methods should be higher than for manipulable methods. Wediscuss philosophy of science impli-

cations, such as f‌ield contingent quality standards and P-value adjustments.

1. introduction

Many proposals for raising hurdles to the publication of experimental results, as in Ben-

jamin et al. (2018), ignore the effect that such proposals will have on experimenter behavior.

Higher quality requirements could discourage potentially valuable experimentation and

encourage manipulation. The net effect could be to lower the value of submitted papers.

Suppose a researcher (sender) can sequentially run private experiments and knows that if

he reveals a surprising outcome that goes against the prior belief, then an editor (receiver)

publishes this outcome. The researcher cares about publication, though less so if the publica-

tion is a result of a false positive. The editor wants to make the correct decision. The sender

can also privately manipulate to achieve the desired outcome. The difference between manip-

ulation and experimentation is that manipulation produces an outcome that is unrelated to an

(unknown) decision-relevant state of the world, whereas an experiment yields an informative

outcome. Manipulation is costly to the sender. In practice, manipulation costs can result from

expected punishment costs and they depend on the research method. For example, manipulat-

ing privately collected data for a regression can hardly be detected and expected punishment

costs are low. If a regression is run on publicly available data instead, then manipulation is

easier to detect and expected punishment costs should be higher.

We study how the face value of the evidence required for publication affects the sender’s

behavior and the quality of the publication process. The face value of a revealed outcome

corresponds to the precision of an experiment that can generate this outcome.2For exam-

ple, when reviewing a paper containing a regression, the editor can assess the quality of the

regression in the manuscript (i) conditional on this regression being run on nonmanipulated

data and (ii) conditional on this specif‌ication being the only regression run. The face value

∗Manuscript received February 2019; revised June 2020.

1Please address correspondence to: Mike Felgenhauer, Brunel University London, Marie Jahoda, Uxbridge, UB8

3PH, UK. E-mail: mike.felgenhauer@brunel.ac.uk.

2Formally, an experiment’s precision is the probability with which its outcome correctly predicts the state of the

world. For a nonmanipulable research method the face value of the revealed outcome is equal to the precision of

the experiment that generated this outcome. For a manipulable method the revealed outcome may either stem from

an experiment with a certain precision or from manipulation, which pretends that it stems from an experiment with

this precision. For a manipulable research method the face value of a revealed outcome is equal to this (actual or

faked) precision.

277

© 2020 The Authors. International Economic Review published by Wiley Periodicals LLC on behalf of the Eco-

nomics Department of the University of Pennsylvania and the Osaka University Institute of Social and Economic Re-

search Association

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, dis-

tribution and reproduction in any medium, provided the original work is properly cited.

278 felgenhauer and xu

of any revealed outcome is observable, by reading the manuscript, but whether the revealed

outcome stems from an experiment or manipulation is not observable. We assume that the

sender can only run experiments with the face value that is required for publication and that

the receiver only publishes a surprising outcome with this face value.3

The article characterizes the face value requirement for publication that maximizes the

quality of the receiver’s decision. We f‌ind that the decision quality may not be maximized with

a demanding high face value. High-quality requirements make it hard to f‌ind a surprising re-

sult that goes against the prior belief via experimentation. Not experimenting or manipulation

may then be more attractive for the sender.

We f‌ind that an increase of the face value may encourage a switch from experimentation

to manipulation or vice versa. In spite of this ambiguity, our major result shows that the face

value requirements for manipulable research methods should be weakly below those for non-

manipulable methods.4In order to illustrate the intuition, suppose that manipulation costs are

suff‌iciently low such that manipulation at the outset is optimal for the sender at the face value

that maximizes the decision quality for nonmanipulable outcomes. The question is how the

face value can be adjusted to make informative experimentation more attractive than uninfor-

mative manipulation. If the sender’s benef‌it from experimentation decreases in the face value,

for example, because he opportunistically cares more for a favorable decision than about the

state of the world, then experimenting becomes more prof‌itable if the face value is reduced:

With a lower face value it is more likely to obtain a surprising experimental outcome by

chance that can be used for persuasion. If the sender’s benef‌it from experimentation increases

in the face value instead, for example, because he cares more about the right decision, then

the face value should be as high as possible for nonmanipulable outcomes. Increasing the face

value for manipulable outcomes in order to encourage experimentation is then not possible.

We argue that there should be different quality standards in scientif‌ic f‌ields that use differ-

ent research methods: The decision quality maximizing publication standards should depend

on experimentation and manipulation costs associated with these methods. We also argue

that a reduction of the default P-value threshold for statistical signif‌icance for claims of new

discoveries, as proposed by 72 authors in Benjamin et al. (2018), may deteriorate the quality

of the publication process by increasing the number of published manipulated articles.

2. literature

This article is part of the persuasion literature in which a sender discloses information to a

receiver who then makes a decision that affects the sender’s well-being (e.g., Jovanovic, 1982;

Milgrom and Roberts, 1986; Glazer and Rubinstein, 2001; Dzuida, 2011). The article belongs

to a branch of this literature that combines persuasion with information acquisition via exper-

imentation.5Many of these papers study public experimentation, where the receiver observes

the experimentation history (e.g., Kamenica and Gentzkow, 2011; Henry and Ottaviani, 2019).

Henry and Ottaviani (2019) f‌ind that it may be in the receiver’s interest to commit to a low

approval standard to achieve desirable stopping behavior of the sender. This is related to our

point that a too challenging face value requirement for persuasion may deter experimentation

and encourage manipulation. Manipulation is not considered in Henry and Ottaviani (2019).

3It can be shown that the results are not affected if the receiver sets the face value requirement and the sender

makes history-dependent precision choices.

4The research method is observable (by reading the manuscript). A method can be viewed as nonmanipulable if

manipulation costs are suff‌iciently high such that manipulation does not occur at any face value. For a manipulable

method, manipulation occurs for some face values. Many papers in the literature assume nonmanipulable methods

and we consider them as an interesting benchmark.

5The article relates to strategic experimentation as in Rothchild (1974), Aghion, et al. (1991), Bolton and Harris

(1990), Keller, et al. (2005), and Rosenberg, et al. (2007). A survey on these “bandit problems” is Bergemann and

Välimäki (2008). Experimentation is also studied in the literature on the classical problem of sequential analysis (as

in Wald, 1947; Moscarini and Smith, 2001).