The development and evaluation of a scale to assess job satisfaction in public universities with item response theory: a Brazilian study

• Date: 21 February 2024
• Pages: 486-503
• DOI: https://doi.org/10.1108/IJPSM-09-2023-0269
• Published date: 21 February 2024
• Subject Matter: Public policy & environmental management,Politics,Public adminstration & management
• Author: Jerko Ledic Neto,Dalton Francisco Andrade,Hai-Yan Helen Lu,Anna Cecilia Mendonca Amaral Petrassi,Antonio Renato Pereira Moro

The development and evaluation

of a scale to assess job satisfaction

in public universities with

item response theory:

a Brazilian study

Jerko Ledic Neto

Department of Production Engineering and Systems,

Federal University of Santa Catarina, Florianopolis, Brazil and

Faculty of Engineering and Information Technology,

University of Technology Sydney, Sydney, Australia

Dalton Francisco Andrade

Department of Statistics and Information Technology,

Federal University of Santa Catarina, Florianopolis, Brazil

Hai-Yan Helen Lu

Faculty of Engineering and Information Technology,

University of Technology Sydney, Sydney, Australia, and

Anna Cecilia Mendonca Amaral Petrassi and

Antonio Renato Pereira Moro

Department of Production Engineering and Systems,

Federal University of Santa Catarina, Florianopolis, Brazil

Abstract

Purpose –This study aimed to devel op a psychometrically r eliable job satisfact ion (JS) measure for

university employee s, guiding administra tive decisions and monit oring satisfaction ove r time in public

universities.

Design/methodology/approach –A JS survey developed by a Brazilian federal university’s sustainability

committeecontaining 58 items across physical, cognitive and organizational domains was longitudinally tested

with 1,214 responses collected. The data were analyzed usingItem Response Theory (IRT) analysis, employing

the Graded Response Model, with tools such as frequency analysis, item characteristic curve, and full-

information factor analysis in RStudio. The scale’s criterion validity was also established via expert qualitative

interpretation.

Findings –The instrument’s internal consistency was confirmed as the results demonstrated its high

reliability with a marginal reliability coefficient of 0.95. Significant findings revealed that recognition and

supervisor relationships were key discriminators of JS and that workers began to perceive satisfaction when

basic environmental conditions were met.

Research limitations/implications –It is important to mention that the application of this scale is

specifically limited to higher education institutions and may not be directly applicable to other educational

settings or industry sectors without modifications.

Originality/value –Although numerous measures and scales have been developed to assess JS, one

elaborated by using IRT in a public university environment was lacking. Due to shifting dynamics in the

workplace, traditional measurement of JS has proven inadequate, necessitating a more precise, accessible

and updated tool. The developed scale allows precisely targeted interventions to improve JS and can be

IJPSM

37,4

486

This study was financed in part by the Coordenaç~

ao de Aperfeiçoamento de Pessoal de N



ıvel Superior -

Brasil (CAPES) - Finance Code 001.

The current issue and full text archive of this journal is available on Emerald Insight at:

https://www.emerald.com/insight/0951-3558.htm

Received 19 September 2023

Revised 15 December 2023

24 January 2024

Accepted 29 January 2024

International Journal of Public

Sector Management

Vol. 37 No. 4, 2024

pp. 486-503

© Emerald Publishing Limited

0951-3558

DOI 10.1108/IJPSM-09-2023-0269

reapplied to evaluate their effectiveness. This research thus contributes a valuable tool for academic

organizational psychology, enhancing the understanding of the measurement of JS.

Keywords Public sector management, Item response theory, University environment, Scale development,

Measurement instruments

Paper type Research paper

Introduction

When workers experience an emotional state termed job satisfaction (JS), the most desirable

outcomes are likely to materialize in terms of performance. Not surprisingly, JS has a long and

fruitful history in the organizational sciences (Stanton et al., 2002). It has been the subject of

extensive study beginning in the early 1900s and continues to be of interest (Glick, 1992). With

increasing recognition of the value of positive organizational behavior, organizations seek to

improve employees’JS by strengthening the psychosocial resources of employees (Pan et al., 2015).

Research has shown that JS influences productivity and effectiveness (Janib et al., 2021;

Montuori et al., 2022;Ngirande, 2021;Sadeghi et al., 2012) by affecting various performance

outcomes, such as the quality of life, stress, burnout, anxiety, turnover intention (Briganti

et al., 2023;Chen and Wang, 2019;Holm et al., 2015;Pan et al., 2015), workplace incivility (Chen

and Wang, 2019;Hur et al., 2015) and absenteeism (Montuori et al., 2022;Olafsen et al., 2017;

Schroder, 2008). Although, the concept of JS is a heavily researched area within industrial

psychology, sociology, and the field of organizational behavior (Al-Zoubi, 2012), definitions of

JS vary between authors (Masum et al., 2015). Nonetheless, Locke (1969)’s definition is still

relevant and widely cited. In his paper called “What is Job Satisfaction?”Locke (1969)

describes JS as the pleasurable emotional state resulting from the appraisal of one’s job as

achieving or facilitating the achievement of one’s job values.

Numerous metrics and scales have been developed to measure JS. The first contemporary

measure of JS, published by Hoppock in 1935, was a 4-item measure of general JS (Stanton

et al., 2002). Some of the most widely used instruments are the Minnesota Satisfaction

Questionnaire (Weiss et al., 1967), the Job Satisfaction Index (Smith, 1969), Michigan

Organizational Assessment Questionnaire Job Satisfaction Subscale (Cammann et al., 1979),

and Job Satisfaction Survey (Spector, 1985). However, the utility of some instruments in

measuring work satisfaction among academicians has been questioned (Conklin and

Desselle, 2007) due to the unique and evolving nature of the university environment.

In general, if characteristics of work situations, differ from those that existed when the

scales were constructed, there may have been a gradual erosion of the scales’measurement

properties (Roznowski, 1989). Devellis and Thorpe (2021) argue that scale validation is a

cumulative, ongoing process, and changes in measurement characteristics over time, known

as item drift (Roznowski, 1989) should be taken into account when developing a reliable and

updated instrument for the university context.

In response to these concerns, the data was analyzed using item response theory (IRT) to

develop an updated and reliable JS instrument tailored to the unique characteristics of the

university environment. To Reise et al. (2005), IRT is a group of mathematical models used to

classify items on a scale, and to evaluate the quality of an item, serving to produce

psychometric properties about the construct. Although classical test theory (CTT) has served

test development well over several decades, IRT has become mainstream as the theoretical

basis for measurement (Embretson and Reise, 2013) and was rapidly embraced by test

manufacturers as a much more realistic model (De Ayala, 2013), allowing for comparisons

between workers, institutions, and over time (Ledic et al., 2022).

The development of a JS metric using data collect in a Brazilian public higher education

institution is motivated by the challenges this kind of institution face. Operating under tight

budget limitations, Brazilian public universities often place si gnificant demands on

A scale to

assess job

satisfaction

487