Toward fast network intrusion detection for web services: partial-flow feature extraction and dataset construction

• Date: 17 December 2024
• Pages: 77-95
• DOI: https://doi.org/10.1108/IJWIS-09-2024-0261
• Published date: 17 December 2024
• Subject Matter: Information & knowledge management,Information & communications technology,Information systems,Library & information science,Information behaviour & retrieval,Metadata,Internet
• Author: Tianfeng Wang,Yingying Xu,Zhenzhou Tang

Toward fast network intrusion detection

for web services: partial-f‌low feature

extraction and dataset construction

Tianfeng Wang,Yingying Xu and Zhenzhou Tang

Wenzhou Key Laboratory for Intelligent Networking,

Wenzhou University, Wenzhou, China

Abstract

Purpose –Timely intrusion detectionin extensive traff‌ic remains a pressing and complex challenge, including

for Web services. Current research emphasizes improving detection accuracy through machine learning, with

scant attention paid to the dataset’s impact on the capability for fast detection. Many datasets rely on f‌low-level

features, requiring entire f‌low completion before determining if it constitutes an attack,reducing eff‌iciency. This

paper aims to introduce a new feature extraction method and construct a new security dataset that enhances

detection eff‌iciency.

Design/methodology/approach –This paper proposes a novel partial-f‌low feature extraction method that

extracts packet-level features eff‌iciently to reduce the high latency of f‌low-level extraction. The method also

integrates statistical and temporal features derived from partial f‌lows to improve accuracy. The method was applied

to the original packet capture (PCAP) f‌iles utilized in creating the CSE-CIC-IDS 2018 dataset, resulting in the

development of the WKLIN-WEB-2023 dataset specif‌ically designed for web intrusion detection. The

effectiveness of this method was evaluated by training nine classif‌ication models on both the WKLIN-WEB-2023

and CSE-CIC-IDS 2018 datasets.

Findings –The experimental results show that models trained on the WKLIN-WEB-2023 dataset consistently

outperform those on the CSE-CIC-IDS 2018 dataset across precision, recall,f1-score, and detection latency. This

demonstrates the superior effectiveness of the new dataset in enhancing both the eff‌iciency and accuracy of

intrusion detection.

Originality/value –This study proposes the partial-f‌low feature extraction method, creating the WKLIN-WEB-

2023 dataset. This novel approach signif‌icantly enhances detection eff‌iciency while maintaining classif‌ication

performance, providing a valuable foundation for further research on intrusion detection eff‌iciency.

Keywords Network intrusion detection, Fast detection, Webattack, Security dataset

Paper type Research paper

1. Introduction

Networkintrusion detectionsystems (NIDS) are crucialfor rapidly identifyingand responding

to the increasing intrusions within network traff‌ic, providing timely threat detection and

defense (Pharate et al., 2015). Given the critical role of web services in modern digital

interactions, web servers have become prime targets forcyber attackers, leadingto signif‌icant

security threats (Vadhil et al.,2024). The surge in network traff‌ic further intensif‌ied web

services’security challenges, making NIDS essential for defending against diverse network

attacks (Wan da a nd Ji e, 2 02 0). Consequently, enhancing intrusion detection capabilities for

web services has become a key research focus in cyberspace security (He et al.,2023).

Funding: This work was supported by the Fundamental Scientif‌ic Research Project of Wenzhou City

(G20220032) and the Natural Science Foundation of Zhejiang Province (LZ20F010008).

International

Journal of Web

Information

Systems

77

Received11 Septembe r 2024

Revised6 November 2024

Accepted11 November 2024

InternationalJournal of Web

InformationSystems

Vol.21 No. 1, 2025

pp. 77-95

© Emerald Publishing Limited

1744-0084

DOI 10.1108/IJWIS-09-2024-0261

The current issue and full text archive of this journal is available on Emerald Insight at:

https://www.emerald.com/insight/1744-0084.htm

In recent years, to enhance the performanceof intrusion detection, numerous studies have

leveraged classif‌ication models from the f‌ield of artif‌icial intelligence for attack detection

(Liu and Lang, 2019), continuously optimizing these models to improve detection accuracy

(Sajith and Nagarajan, 2022;Alqarni, 2023;Alzaqebah et al., 2023). At the same time, to

reduce detection latency, researchers have introduced methods such as feature selection (Di

Mauro et al., 2021), model deployment optimization(Saleh et al., 2019), and distributed

parallel computing(Wu et al., 2022).However, a literature review reveals that while existing

research has concentrated on optimizing models and computational methods, the impact of

the dataset itself on the eff‌iciency of intrusiondetection has been largely overlooked.

Currently, the datasets used for intrusion detection primarily rely on f‌low-level features

rather than packet-level features (Ring et al.,2019). Packet-level features are extracted from

individual packets within a f‌low and allow for real-time attack detection based on the

information from the current packet. However, this approach fails to capture the relationships

between packets, which reduces detection accuracy. In contrast, f‌low-level features, extracted

after the entire f‌low is transmitted, provide a more comprehensive view by incorporating

temporal and contextual information, signif‌icantly improving detection accuracy (Rodríguez

et al.,2022). Despite this, this approach introduces considerable latency due to the need to wait

for the entire f‌low to be transmitted.

This dilemma raises criticalquestions:

Q1. Is it necessary to wait for the complete f‌low to detectintrusions effectively?

Q2. Can we extract features from partial packets within an ongoing f‌low, thereby

reducing detectionlatency while maintaining or even enhancing detection accuracy?

Addressing these questionsis the central focus of our research.

Considering the above factors, we propose a partial-f‌low feature extraction method that

integrates both packet-level and f‌low-level features. This approach eff‌iciently extracts packet-

level features, overcoming the high latency associated with traditional f‌low-level feature

extraction. By incorporating statistical and temporal features from partial f‌lows, the accuracy of

packet-level features is further enhanced. We apply this method to the original Web PCAPf‌iles

used in the generation of the CSE-CIC-IDS 2018 dataset, creating the new security dataset,

WKLIN-WEB-2023. Comprehensive experiments were conducted to evaluate the effectiveness

of the proposed method and the adaptability of the new dataset for intrusion detection. The main

contributions of this paper are as follows:

The WKLIN-WEB-2023 dataset we constructed proves suitabl e for training and testing

a diverse range of intrusion detection classif‌ication models, including but not limited to

LightGBM, XGBoost, Random Forest (RF), Multilayer Perceptron (MLP), Decision

Tree (DT), CatBoost, Logistic Regression (LR), Convolutional Neural Network (CNN)

and ResNet50.

Experimental results demonstrate that it is unnecessary to wait for the completion of an

entire f‌low transmission to detect an attack. Notably, utilizing only one-third of the

packets within a f‌low is suff‌icient to determine the presence of an intrusion accurately.

The proposed partial-f‌low feature extraction method enables faster intrusion detection,

narrowing the opportunity window for an attacker to cau se harm. Experimental data

show that this method reduces detection time by an average of 20.71 s per f‌low.

The remainder is organizedas follows. Section 2 introduces commonly useddatasets, related

research, and analysis of our work. Section 3 details the partial-f‌low feature extraction

IJWIS

21,1

78