Conv-DMSA: an efficient imputation model for multivariate time series through diagonal mask self-attention

• Date: 26 December 2024
• Pages: 22-36
• DOI: https://doi.org/10.1108/IJWIS-04-2024-0119
• Published date: 26 December 2024
• Subject Matter: Information & knowledge management,Information & communications technology,Information systems,Library & information science,Information behaviour & retrieval,Metadata,Internet
• Author: Hao Zhang,Weilong Ding,Qi Yu,Zijian Liu

Conv-DMSA: an eff‌icient imputation

model for multivariate time series

through diagonal mask self-attention

Hao Zhang,Weilong Ding,Qi Yu and Zijian Liu

North China University of Technology, Beijing, China and Beijing Key Laboratory

on Integration and Analysis of Large-Scale Stream Data, Beijing,China

Abstract

Purpose –The proposed model aims to tackle the data quality issues in multivariatetime series caused by

missing values. It preservesdata set integrity by accurately imputing missing data, ensuringreliable analysis

outcomes.

Design/methodology/approach –The Conv-DMSA model employs a combination of self-attention

mechanisms and convolutional networks to handle the complexities of multivariate time series data. The

convolutional network is adept at learning features across uneven time intervals through an imputation feature map,

while the Diagonal Mask Self-Attention (DMSA) block is specif‌ically designed to capture time dependencies and

feature correlations. This dual approach allows the model to effectively address the temporal imbalance, feature

correlation and time dependency challenges that are often overlooked in traditional imputation models.

Findings –Extensive experiments conducted on two public data sets and a real project data set have demonstrated

the adaptability and effectiveness of the Conv-DMSA model for imputing missing data. The model outperforms

baseline methods by signif‌icantly reducing the Root Mean Square Error (RMSE) metric, showcasing its superior

performance. Specif‌ically, Conv-DMSA has been found to reduce RMSE by 37.2% to 63.87% compared to other

models, indicating its enhanced accuracy and eff‌iciency in handling missing datain m ultivariatetime series.

Originality/value –The Conv-DMSA model introduces a unique combination of convolutional networks

and self-attention mechanisms to the f‌ield of missingdata imputation. Its innovative use of a diagonal mask

within the self-attention blockallows for a more nuanced understanding of the data’s temporal and relational

aspects. This novel approach not only addresses the existing shortcomings of conventional imputation

methods but also sets a newstandard for handling missing data in complex, multivariatetime series data sets.

The model’s superior performance and its capacity to adapt to varying levels of missing data make it a

signif‌icantcontribution to the f‌ield.

Keywords Diagonal mask self-attention, Multivariate time series, Convolutional network,

Data imputation

Paper type Research paper

1. Introduction

Multivariate time series (Bignoumba et al.,2024) exist extensively in practical applications,

such as stock prices, traff‌icf‌low, etc. Such data usually have inherent feature dependencies

and temporal correlations. Data missing is a common phenomenon, usually caused by the

inability to obtain or damage of data points due to various faults during the data collection

process. Such data incompleteness poses diff‌iculties and challenges for data analyses.

Specif‌ically, missing data disrupts the data set’s continuity, thereby undermining its overall

reliability and validity. Therefore, it’s crucial to consider its data quality, and how to

eff‌iciently handle such missing data for multivariate time series has great signif‌icance in

practice. Data imputation is one effective solution to guarantee complete data set, and

The work is supported by the YuxiuInnovation Project of NCUT (2024NCUTYXCX102).

IJWIS

21,1

22

Received22 April 2024

Revised15 July 2024

6 September 2024

Accepted6 Se ptember 2024

InternationalJournal of Web

InformationSystems

Vol.21 No. 1, 2025

pp. 22-36

© Emerald Publishing Limited

1744-0084

DOI 10.1108/IJWIS-04-2024-0119

The current issue and full text archive of this journal is available on Emerald Insight at:

https://www.emerald.com/insight/1744-0084.htm