An evaluation of Web of Science,
Scopus and Google Scholar
citations in operations management
University of Alabama, Tuscaloosa, Alabama, USA, and
Alexander E. Ellinger
Department of Marketing, University of Alabama, Tuscaloosa, Alabama, USA
Purpose –Ongoing deliberation a bout how research product ivity should be measured is e xacerbated
by extensive disparity b etween the number of citatio ns for scholarly works rep orted by commercial
academic search engin es and Google Scholar (G S), the premier web crawl ing service for discove ring
research citations. D isparities identified in citat ion comparison studies have also l ed to disagreement about
the value of the higher number of citat ions for social sciences and business scholarly artic les consistently
reported by GS. The purp ose of this paper is to extend previous databa se citation comparison studies by
manually analyzing a sa mple of unique GS citatio ns to a leading operatio ns management journa l (i.e.
citations found only in GS a nd not the commercial sea rch engines) to reveal jus t where these additional
citations are coming fr om.
Design/methodology/approach –In addition tocomparing citation counts for the threedatabases, unique
GS citationdata for the sample of journal articleswas manually captured and reviewed.The authors’approach
providesa much more in-depth examinationof the provenance of GS citationsthan is found in previousstudies.
Findings –The findings suggest that co ncerns about the value of unique G S citations may not be
warranted since the documen t types for the unique GS citing docum ents identified in the analys is are
dominated by familia r scholarly formats. Pr edominantly authent ic and validated journ al publications,
dissertations, conference papers, and book and book chapters accounted for the large majority of the
unique GS citations ana lyzed.
Practical implications –The study lends further credence to contentions that the use of citations reported
in GS is appropriate for evaluating research impact in disciplines where other formats beyond the
English-language journal article are valued.
Originality/value –Developing a more informed understanding of the provenanceof unique GS citations in
the authors’field is important because many scholars not only aspire to publish in elite journals with high
impact factors based on citation counts provided by commercial databases to demonstrate quality, but also
report the larger number of citations for their publications that are reported by GS to demonstrate impact.
The in-depth manual analysis suggests that GS provides a more nuanced and comprehensive representation
of research impact and international scope than the commercial databases.
Keywords North America, Survey, Process management, Performance measurements
Paper type Research paper
Scholarly metrics are being used to assess faculty research productivity to the extent that
“academic research is being parameterized as never before”(McKinnon, 2017, p. 431).
As faculty research output, quality and impact are increasingly scrutinized, there continues
to be considerable deliberation about how faculty research productivity should be
measured. This ongoing debate is exacerbated by the availability of frequently inconsistent
citation counts for scholarly works in the Web of Science (WoS), Scopus and Google Scholar
(GS) databases. More specifically, there is extensive disparity between the number of
citations for scholarly works reported by traditional academic search engines like WoS and
Scopus and GS, the premier web crawling service for discovering citations to academic
research. The consistent, extensive disparities identified in citation comparison studies has
led to disagreement about the relative value of the various citation counts for scholarly
research reported by these databases.
The International Journal of
Vol. 30 No. 4, 2019
© Emerald PublishingLimited
Received 12 April 2019
Revised 7 June 2019
21 July 2019
Accepted 22 July 2019
The current issue and full text archive of this journal is available on Emerald Insight at: