Graph-based rank aggregation: a deep-learning approach

• Date: 22 November 2024
• Pages: 54-76
• DOI: https://doi.org/10.1108/IJWIS-09-2024-0278
• Published date: 22 November 2024
• Subject Matter: Information & knowledge management,Information & communications technology,Information systems,Library & information science,Information behaviour & retrieval,Metadata,Internet
• Author: Amir Hosein Keyhanipour

Graph-based rank aggregation:

a deep-learning approach

Amir Hosein Keyhanipour

Computer Engineering Department, Faculty of Engineering,

College of Farabi, University of Tehran, Tehran, Iran

Abstract

Purpose –This study aims to introduce a novel rank aggregation algorithm that leverages graph theory and

deep-learning to improve the accuracy and relevanceof aggregated rankings in metasearch scenarios, particularly

when faced with inconsistent and low-quality rank lists. By strategically selecting a subset of base rankers, the

algorithm enhances the quality of the aggregatedranking while using only a subset of base rankers.

Design/methodology/approach –The proposed algorithm leveragesa graph-based model to represent the

interrelationships between base rankers.By applying Spectral clustering, the algorithm identif‌ies a subset of

top-performing base rankersbased on their retrieval effectiveness. These selected rankersare then integrated

into a sequentialdeep-learning model to estimate relevancelabels for query-document pairs.

Findings –Empirical evaluation on the MQ2007-agg and MQ2008-agg data sets demonstrates the substantial

performance gains achieved by the proposed algorithm compared to baseline methods, with an average

improvement of 8.7% in MAP and 11.9% in NDCG@1. The algorithm’s effectiveness can be attributed to its ability

to effectively integratediverse perspectives from base rankers and capture complex relationships within the data.

Originality/value –This research presentsa novel approach to rank aggregationthat integrates graph theory

and deep-learning.The author proposes a graph-based model toselect the most effective subset for metasearch

applications by constructing a similarity graph of base rankers. This innovative method addresses the

challengesposed by inconsistent and low-quality rank lists, offeringa unique solution to the problem.

Keywords Rank aggregation, Deep-learning, Graph theory

Paper type Research paper

1. Introduction

Rank aggregation, a cornerstone in information retrieval and machine learning, entails the

synthesis of multiple ranked lists into a unif‌ied consensus list. This process is particularly

imperative when confronted with disparate and often conf‌licting ranking sources, such as

search engines, recommendation systems, medical diagnoses, anomaly detection and fraud

detection. Effective rank aggregation techniques aspire to generate a consensus list that

accurately mirrors theunderlying preferences or relevance of the ranked items.

Compliance with ethical standards.

Ethical and informed consent for data used: This study does not make use of any personal data and

therefore does not require anyone’s informed consent.

Disclosure of potential conf‌licts of interest: The authors have no competing interests to declare that

are relevant to the content of this article.

Authors contribution statement: The author, conf‌irms sole responsibility for the following: study

conception and design, data collection, analysis and interpretation of results and manuscript

preparation.

Data availability and access: All the data sets used in this manuscript are published and publicly

available for research. References to data sources are provided in the manuscript.

Funding declaration: This research received no specif‌ic grant from any funding agency in the

public, commercial or not-for-prof‌it sectors.

IJWIS

21,1

54

Received27 September 2024

Revised14 Oc tober2024

19 October2024

Accepted30 October 2024

InternationalJournal of Web

InformationSystems

Vol.21 No. 1, 2025

pp. 54-76

© Emerald Publishing Limited

1744-0084

DOI 10.1108/IJWIS-09-2024-0278

The current issue and full text archive of this journal is available on Emerald Insight at:

https://www.emerald.com/insight/1744-0084.htm

Broadly speaking,existing rank aggregation algorithms can be classif‌ied into two primary

categories: order-based and score-based algorithms (Li, 2015). Order-based algorithms

generate a unif‌ied ranked list by leveraging the relative order of the individual ranked lists

(D’Ambrosio et al., 2019;Deng et al., 2014;Gu and Yu, 2023;Martinez-Muñoz et al.,

2009), while score-based algorithms use the scores assigned to items by local rankers

(Barman and Shah, 2017a;Boehmer et al., 2023;Ciaccia and Martinenghi, 2018;Jamadi

Khiabani et al., 2019). Both categories of rank aggregation algorithms can benef‌it from

supervised, unsupervised or semisupervised learning techniques. This research focuses on

proposing a novel order-based rank aggregation algorithm, which is particularly applicable

in metasearch scenarios where only rank information from individual search engines is

available. However, aggregating often inconsistent and low-quality rank lists presents a

formidable challenge. Existing rank aggregation algorithms are often computationally

expensive due to their reliance on the entire data set and all base rankers. In addition, the

heterogeneous quality of base rankers can negatively impact the quality of the aggregated

ranking. Interrelations ofbase rankers are not included in the aggregation process of mostof

the existing methods. To address these limitations, this researchproposes a novel framework

that uses graph theory to captureinterrelationships between base rankers. By clustering base

rankers, the proposed framework reduces the computational burdenof the rank aggregation

process while ensuring the inclusion of the most relevant rankers. These nominated rankers

are then integrated into a sequential deep-learning model to predict the relevance of

documents. The proposed approach offers several advantages over traditional rank

aggregation methods. It offers computational eff‌icacy due to the reduced number of

rankers involved, and it improves the quality of the aggregated ranking by considering

interrelationships betweenrankers. Furthermore, the use of a deep-learning model allows for

the capture of complex relationships,leading to more accurate and informative rankings.

Specif‌ically, motivated by the success of the graph model in handling learning to rank

problem (Keyhanipour,2023, 2024;Yeh and Tsai, 2022) as we as the powerful applications

of deep-learning algorithmsin the rank aggregation context (Hu et al., 2023;Kouadria et al.,

2020;Wang, 2024;Wang et al., 2022;Xu et al., 2021), this paper presents a novel rank

aggregation approach which integrates the capabilities of the graph models as well as deep-

learning techniques in rank aggregation realm. Specif‌ically, this graph-based representation

of the used rank aggregation data set, enablesus to model the interrelationships between base

rankers and incorporatethem into the aggregation process. By applying Spectral clustering to

this graph and selecting a subset of base rankers from each cluster based on their retrieval

performance, we enhance both the quality of the rank aggregation output and the overall

eff‌icacy of our proposed approach.To further ref‌ine our algorithm, we introducea weighting

scheme for base rankers that considers their retrieval effectiveness, enabling us to bypass

low-quality rankers. The nominated rankers are then fed into a sequential deep-learning

model, which constructs a relevanceestimation model capable of estimating the relevance of

any given query-documentpair based on their ranking in the local ranked lists of the selected

base rankers. Sequential deep-learning models are particularly well-suited for rank

aggregation tasks because they effectivelyrepresent the ordinal relationships between items

in a ranking list and capture temporal dependencies. This model allows us to effectively

aggregate local rankings and produce a unif‌ied ranked listas the output of our proposed rank

aggregation algorithm.

To validate the eff‌icacy of our proposed algorithm, we conducted experiments on two

major real-world data sets, MQ2007-agg and MQ2008-agg (Liu et al.,2007). Experimental

results demonstrate that our algorithm outperforms baseline and state-of-the-art rank

aggregation methodsaccording to the evaluation criteria.

International

Journal of Web

Information

Systems

55