Manaar Alam

Backdoor attacks enable adversaries to implant hidden triggers in machine learning models, causing targeted malicious behavior while maintaining normal performance. These attacks pose serious security risks in applications ranging from autonomous systems to financial decision-making. However, detecting backdoors is challenging due to intellectual property (IP) protections that restrict access to model internals, limiting forensic analysis. This talk will explore a detection strategy based on system profiling, analyzing execution characteristics such as memory usage, CPU activity, and cache behavior to identify anomalies without requiring direct model inspection. Backdoor attacks in federated learning (FL) present additional challenges, as decentralized training prevents adversaries from exerting direct control over the global model, while aggregation naturally weakens malicious updates. This talk will examine how adversaries can overcome these challenges by strategically reinforcing backdoors across multiple training rounds to ensure persistence despite aggregation. Furthermore, we will explore a stealth technique that allows adversaries to selectively remove backdoors to evade detection and reintroduce them later, making traditional defenses ineffective. The discussion will highlight the evolving nature of backdoor threats and the need for robust countermeasures in both centralized and federated learning environments.

Federated Learning (FL) enables multiple participants to train deep learning models collaboratively without exposing sensitive personal data. However, its distributed nature and unvetted data expose it to the potential threat of backdoor attacks. Adversaries exploit this vulnerability to inject malicious functionality into the centralized model during training, causing intentional misclassifications for specific adversary-chosen inputs. In this talk, we delve into the dual aspects of a backdoor attack: its creation and subsequent removal when deemed necessary by the adversary. The first facet pertains to a method of persistent-by-construction backdoor injection developed for FL. It leverages adversarial perturbation and selectively targets specific parameters of the centralized model. However, the persistent nature of these backdoors can be double-edged, prompting prevention measures. Hence, it is critical for adversaries to remove these backdoors either after the successful achievement of their intended objectives or in response to suspected detection attempts. In view of this, the second facet of our discussion extends the concept of machine unlearning to effectively remove these persistent backdoors from the centralized model. It outlines strategies to maintain the performance of the centralized model and prevent over-unlearning of information unrelated to backdoor patterns, enhancing the stealth of adversaries during backdoor removal. Collectively, these two strategies create a dynamic and flexible adversarial approach to backdoor attacks and removal in the context of FL.

In this part of the talk, we provide a detailed overview on both the boon and bane of AI on Security. To be more specific we start with describing how Machine Learning (ML)/Deep Learning (DL) can be leveraged to perform advanced side channel attacks on cryptographic implementations. Subsequently, we present deep learning based methodologies for leakage assessment due to fault attacks on crypto-devices. We further present a state-of-the-art overview on the threats of machine learning in modeling Physically Unclonable Functions (PUFs), a promising hardware security primitive. Subsequently, we look at the opportunities from DL based methods in developing effective diagnostic tools for powerful malwares. We present case-studies on using Performance Counter based approaches in detecting menacing threats like ransomware and rowhammer attacks.

True Random Number Generators (TRNGs) are one of the most crucial components in the design and use of cryptographic protocols and communication. Predictability of such random numbers are catastrophic and can lead to the complete collapse of security, as all the mathematical proofs are based on the entropy of the source which generates these bit patterns. The randomness in the TRNGs is hugely attributed to the inherent noise of the system, which is often derived from hardware subsystems operating in an ambiguous manner. However, most of these solutions need an add-on device to provide these randomness sources, which can lead to not only latency issues but also can be a potential target of adversaries by probing such an interface. In this talk, we will see how to alleviate these issues by proposing an in-situ TRNG construction, which depends on the functioning of the underlying hardware architecture. These functions are observed via the Hardware Performance Counters (HPCs) and are shown to exhibit high-quality randomness in the least significant bit positions. We provide extensive experiments to research on the choice of the HPCs, and their ability to pass the standard NIST and AIS 20/31 Tests. We also analyze a possible scenario where an adversary tries to interfere with the HPC values and show its effect on the TRNG output with respect to the NIST and AIS 20/31 Tests. Additionally, to alleviate the delay caused for accessing the HPC events and increase the throughput of the random-source, we also propose a methodology to cascade the random numbers from the HPC values with a secured hash function.