Blog Post

Security, Trust and Privacy in 5G/6G/Beyond with AI/ML Integrations

  • Vahid Maleki Raee
  • published date: 2026-02-06 11:42:32

This blog post explores the integration and impact of Artificial Intelligence and Machine Learning (AI/ML) on security in wireless communications—particularly within 5G, 6G, and beyond. While the broader context includes security within the 6G architecture and emerging 6G technologies [1], this post specifically focuses on pre‑6G security.

Pre-6G Security:

6G is the next generation communication network that is considered a promising technology to advance wireless communications services and satisfy the future network communication demands by 2030 [2]. Before diving into the 6G technology, it is important to understand the main characteristics of the 5G network.

In nutshell, the primary focus of 5G has been to satisfy quality-of-service (QoS) requirements of industry verticals, which is usually characterized by

  • Enhanced mobile broadband (eMBB) to address high-data-rate access,

  • Ultra-reliable low-latency communications (URLLC) supporting mission-critical, latency-sensitive applications, and

  • Massive machine-type communications (mMTC) to realize dense connectivity of Internet of Things (IoT) devices.

In addition, the 5G network enables the network softwarization and automation that is introduced by Software-defined networking (SDN), followed by network virtualization that came into practice by Network Function Virtualization (NFV) and Mobile Edge Computing (MEC) to bring the cloud computing capabilities closer to edge of the network, closer to the end users, to improve the processing latency.

On the other hand, the security has always been a main concern in wireless communications. To be more specific, the security in pre-6G paradigm falls into the security of SDN, MEC, constrained network. By advancements in AI/ML techniques and algorithms, the AI can be used in two ways: one it can be used as a tool for defense, or it can be used for adversarial attacks in communication systems. As wireless communication network is concerned, ML techniques can be widely applied for anomaly-based intrusion detection, or the communication link attributes and user behavior can be used with ML for authentication and authorization purposes.

By definition, the Intrusion Detection Systems (IDS) are used to monitor the network traffic and identify and detect malicious activities [3]. Moreover, the Software-Defined Networking (SDN) is a network architecture developed to softwarized the network, making it programmable, flexible and agile by separating the control plane and data plane. The logically centralized programmable SDN controllers will host the network intelligence, configurations and operation through software applications [4]. There is always a need to have an efficient, adaptive and scalable IDS to protect an SDN architecture against known and unknown security threats. Furthermore, the solution needs to be optimized in terms of accuracy and delay. To cope with this requirement, one possible approach can be to apply Deep Reinforcement Learning (DRL) in conjunction with the Long Short-Term Memory Network (LFTS-RNN) and Particle Colony-Adjusted Jumping Teaching Fish Algorithm Optimization Algorithm (PC-JTFOA) may facilitate the efficiency and flexibility of IDS in SDNs, aiming at enhancing the detection accuracy, reducing false positive, and enhancing the efficiency in an acceptable response time [4].

An overview of DRL-based IDS [3]

Furthermore, considering the Mobile Edge Computing (MEC) networks, the Large Model (LM), as an emerging technique, has gain attention. However, it is difficult to be deployed in resource-constrained MEC devices. Although it is possible to compress the large models into tiny ones, but they get impacted by data drift (the growth of data traffic, diversity, and complexity, leads to have different data samples as compared to original training data, impacting the accuracy in real-time anomaly detection systems). One way to address this challenge is apply a multi-model anomaly traffic detection framework called Continuous Learning for Detection (CL4Det), where the tiny models deployed in MEC networks can achieve a comparable performance.

Multi-model Anomaly Traffic Detection with LM-powered Continuous Learning [5]

At a high-level description, the task-specific models (tiny models) are deployed at the edge servers, and they are continuously getting the knowledge from the large model that is deployed in the cloud. This possibly improves the accuracy while reduces the training overhead. To this end, the model requires to be evaluated periodically to detect the data drift (performance degradation). Then, it is required to apply retraining by selecting the training configurations, and finally, it is indeed needed to schedule the training and detection tasks to achieve both high accuracy and completing all training tasks accordingly [5].

Architecture of CL4Det [5]

 

Conclusion:

To conclude, the anomaly detection has always been an interesting and emerging topic of interest in cybersecurity. Considering complex multi-edge and SDN network scenarios, more diverse/complex/large dataset need to be collected and more real-time fine-designed mechanisms worth investigating, while taking into account the limitation of constrained environment.

Edited By: Windhya Rankothge, PhD, Canadian Institute for Cybersecurity 

References:

  1. https://www.youtube.com/watch?v=xlFSdkkdx4A

  2. Jiang, W., Han, B., Habibi, M. A., & Schotten, H. D. (2021). The road towards 6G: A comprehensive survey. IEEE Open Journal of the Communications Society2, 334-366. https://ieeexplore.ieee.org/document/9349624

  3. Kanimozhi, R., & Ramesh, P. S. (2025). Deep reinforcement learning-based intrusion detection scheme for software-defined networkingScientific Reports15(1), 38827. http://nature.com/articles/s41598-025-24869-w

  4. https://www.cisco.com/c/en_ca/solutions/software-defined-networking/overview.html#~stickynav=4

  5. Zhang, J., Chen, Z., Cheng, H., Li, J., & Min, G. (2025). Improving Multi-Model Anomaly Traffic Detection in MEC Networks With Large-Model-Powered Continuous LearningIEEE Network. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10847747

#6GSecurity #TelecomSecurity #SecureCommunications #FutureNetworks #5G #SoftwareDefinedNetworking #CybersecurityResearch #SecurityEngineering