Evaluative Practices in Qualitative Management Research: A Critical Review

• Author: Catherine Cassell,Gillian Symon,Phil Johnson
• Published date: 01 January 2018
• DOI: http://doi.org/10.1111/ijmr.12120
• Date: 01 January 2018

International Journal of Management Reviews, Vol. 20, 134–154 (2018)

DOI: 10.1111/ijmr.12120

Evaluative Practices in Qualitative

Management Research: A Critical Review

Gillian Symon, Catherine Cassell1and Phil Johnson2

School of Management, Royal Holloway University of London, Egham TW20 0EX, Surrey, UK, 1Leeds University

Business School, Maurice Keyworth Building, University of Leeds, Leeds LS2 9JT, UK, and 2SheffieldUniversity

Management School, Conduit Road, Sheffield S10 2TN, UK

Corresponding author email: Gillian.Symon@rhul.ac.uk

This paper critically reviews commentaries on the evaluation and promotion of quali-

tative management research. The review identifies two disjunctures: between method-

ological prescriptions for epistemologically diverse criteria and management journal

prescriptions for standardized criteria; and between the culturally dependent pro-

duction of criteria and their positioning in editorials and commentaries as normative

and objective. The authors’ critical social constructionist analysis surfaces underlying

positivist assumptions and institutional processes in these commentaries, which they

argue are producing (inappropriate) homogeneous evaluation criteria for qualitative

research, marginalizing alternativeperspectives, and disciplining individual qualitative

researchers into particular normative practices. The authors argue that interventions

to encourage more qualitative research need to focus as much on editorial, disciplinary

and institutional practices as the practices of individual researchers,and they make rec-

ommendations for changes that may allow qualitative management research to develop

in a more supportive context byrecognizing philosophical diversity as legitimate.

Introduction

Qualitative management research is an umbrella

term for a range of approaches to research (e.g.

Eriksson and Kovalainen 2008; Myers 2012; Symon

and Cassell 2012) that draw on a variety of episte-

mologies (e.g. Duberley et al. 2012), including crit-

ical theory, postmodernism and interpretivism. This

diversity is seen to be one of its strengths (Bluhm

et al. 2011), but also opens up the issue of how such

work can be fairly evaluated(Bettis et al. 2015). Con-

sequently, a variety of potential evaluation criteria

and recommendations for best practice for qualita-

tive management research have been outlined (e.g.

Bansal and Corley 2011; Pratt 2009). Our purpose

here is not to add more criteria to this mix. Rather, we

want to criticallyreview the criteria and recommenda-

tions for best practice already proposed, drawing out

general themes concerning evaluation processes, and

highlighting the implications of current strategies for

encouraging qualitative research in the management

discipline.

We approach this review of evaluative practices

from the position that knowledge is socially con-

structed (Alvesson and Karreman, 2001) and see

this perspective as providing novel insights into the

continuing debates about what constitutes legitimate

qualitative research. While other commentaries on

the nature of management research in a changing

culture may be based on underlying critical and so-

cial constructionist perspectives (e.g. Alvesson and

Sandberg 2012; Grey 2010), the specific debate on

quality criteria for qualitative management research

has remained largely immune to these perspectives,

relying more on a skills-based or empiricist account

of knowledge. However, we suggest that it is differ-

ent philosophical stances on knowledge production

that lie at the heart of the difficulties of providing

quality criteria for qualitative research. On the one

hand is an understanding that knowledge exists ‘out

there’ and can be discovered, objectively tested or

C2016 British Academy of Management and John Wiley & Sons Ltd. Publishedby John Wiley & Sons Ltd, 9600 Garsington

Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA

Qualitative Management Research 135

verified and, on the other, that knowledge is socially

produced, ‘provisional, mediated, situated [and] con-

tested’ (Blackler et al. 1998, p. 76). So while many

commentators are trying to encourage qualitative re-

searchers to conform to sets of quality criteria that

appear to be generally applicable to all forms of qual-

itativeresearch, they are doing so based on a particular

view of knowledge production that sits uneasily with

the philosophical stances embedded in some types of

qualitative research.

In investigating how qualitative management re-

search has been pursued and discussing how quali-

tative research should be conducted, we need to be

aware of the broader processes by which systems

of knowledge production justify particular practices

and marginalize others. From the reviewof evaluative

practices that forms the basis of this paper, we wantto

provide an alternative perspective on the challenges

of publishing diverse forms of research that acknowl-

edges the wider social construction and institutional-

ization of academic practices. In other words, wewant

to moveaway from a concentration on what individual

qualitative researchers could ‘do better’ to recognize

that they are not the only actors in the network of

relations that is management research.

The paper is structured as follows. First, weprovide

a review of the ‘criteriology debate’ (Schwandt1996)

within the methodological community. This ongoing

debate concerns whether it is feasible or appropriate

to produce generalized evaluative criteria for quali-

tative research. As an outcome of this review, we ar-

gue for the importance of contingent criteria that can

encompass the diversity of the research conducted

under the umbrella term ‘qualitative research’. Sec-

ondly,through an analysis of reviews of methodolog-

ical diversity within management journals and jour-

nal editorials, we analyse how this debate has played

out in the management field specifically. From this

we identify a fundamental disjuncture between the

framing of the qualitative criteriology debate in the

methodological community and that of management

research commentators. Wedemonstrate how the con-

tingency advocated by the former is at odds with the

standardization of the latter.

Consequently, we then explore possible expla-

nations of this disjuncture within the institutional

practices of the management research community.

Reviewing empirical and critical accounts of man-

agement research practice, we demonstrate how the

production of criteria is culturally and temporally

dependent. This analysis exposes a second disjunc-

ture between evidence of quality criteria as subjective

and socially constructed and their positioning as

objective and normative by management commenta-

tors. We see this latter positioning as itself a function

of cultural and institutional pressures towards

standardization. We argue that we also need to pay

attention to these types of processes and practices

to bring about change. Subsequently, we consider

what changes may be required that enable the

identification of excellent qualitative management

research without contorting the distinctive nature

or diversity of qualitative research. Finally, we also

consider the implications of our arguments for the

future of criteriology in management research.

Our aim is to disrupt hegemonic discourse, prompt-

ing reflexivity and debate so as to militate against

movestowards standardization despite the known plu-

rality of qualitative research. We want to draw atten-

tion to the processes of institutionalization that shape

these strategies, and to encourage the consideration of

alternative strategies that do not focus solely on the

individual researcher and that might encourage the

publication of a range of diverse types of qualitative

research, even in the most prestigious management

journals.

Criteriology debates in qualitative

resea rch

When any evaluation of management research is un-

dertaken, criteria of some kind are implicitly, or ex-

plicitly,deployed (Savall et al. 2008). Indeed, evalua-

tion criteria form a boundary without which it would

be difficult to prevent poor quality, untrustworthy or

even illegitimate work from entering the mainstream.

However, any evaluation is a somewhat precarious

process fraught with epistemological ambiguities –

even though it may be often, by default, tacitly pre-

sented as a relatively non-contentious deployment of

benchmarks grounded in a consensus (e.g. Spencer

et al. 2003). Consequently, it is not surprising to find

some debate about the relevance of quality criteria

to qualitative research in the general methodology

community. Sparkes (2001) summarizes the different

approaches to criteriology in the qualitative method-

ology literature as: replication (imitating conventional

(quantitative) validity criteria); parallel (adaptations

of conventional criteria); diversification (accepting a

range of different criteria as suitable to different ap-

proaches); and letting go (a more radical position

that advocates the rejection of any sort of criteria

based on validity claims in favour of criteria based

C2016 British Academy of Management and John Wiley & Sons Ltd.