Towards Quantum Machine Learning for Malicious Code Analysis

GitHub article Paper Link

Abstract

Classical machine learning (CML) has been exten- sively studied for malware classification. With the emergence of quantum computing, quantum machine learning (QML) presents a paradigm-shifting opportunity to improve malware detection, though its application in this domain remains largely unexplored. In this study, we investigate two hybrid quantum- classical models — a Quantum Multilayer Perceptron (QMLP) and a Quantum Convolutional Neural Network (QCNN), for malware classification. Both models utilize angle embedding to encode malware features into quantum states. QMLP captures complex patterns through full qubit measurement and data re- uploading, while QCNN achieves faster training via quantum convolution and pooling layers that reduce active qubits. We evaluate both models on five widely used malware datasets — API-Graph, EMBER-Domain, EMBER-Class, AZ-Domain, and AZ-Class, across binary and multiclass classification tasks. Our results show high accuracy for binary classification — 95–96% on API-Graph, 91–92% on AZ-Domain, and 77% on EMBER-Domain. In multiclass settings, accuracy ranges from 91.6–95.7% on API-Graph, 41.7–93.6% on AZ-Class, and 60.7–88.1% on EMBER-Class. Overall, QMLP outperforms QCNN in complex multiclass tasks, while QCNN offers improved training efficiency at the cost of reduced accuracy

Introduction

Quantum Machine Learning (QML) is an emerging, interdisciplinary field of research that seeks to solve complex problems across a wide range of domains by leveraging the power of quantum computing.Most QNN models have primarily focused on MNIST image classification. MNIST provides benchmarks for testing models that are relatively easier to encode into quantum circuits. However, due to the current hardware limitations, the full 28 × 28 images cannot be encoded directly and have to be downsized to more feasible dimensions, such as 4 × 4 in order to fit the number of qubits used in a quantum circuit. We have implemented both Quantum Multilayer Percep- tron (QMLP) and Quantum Convolutional Neural Network (QCNN) architectures within the PennyLane simulator.

Contributions

The contribution of this paper are as follows:

1. We extensively evaluated Quantum Multilayer Perceptron (QMLP) and Quantum Convolutional Neural Network (QCNN) models across five diverse malware datasets, encompassing both multi-class and binary classification scenarios.
2. This study provides an in-depth analysis of how the architectural design choices of QMLP and QCNN impact their accuracy and overall performance in malware classification across various datasets.
3. We observe that the QCNN model, by leveraging con- volution and pooling mechanisms, offers improved time efficiency. However, this comes at the cost of reduced accuracy across most datasets except for the API-Graph dataset, where QCNN slightly outperforms QMLP in multiclass classification.

Methodology

This section details the experimental setup used to im- plement and evaluate the proposed Quantum Convolutional Neural Network (QCNN) and Quantum Multilayer Perceptron (QMLP). We evaluate the performance of QCNN and QMLP models using five large-scale malware datasets — EMBER- Domain, EMBER-Class, AZ-Domain, AZ-Class, and API-Graph in both multiclass and binary classification settings. Our methodological approach encompasses three main phases: data preprocessing, hybrid quantum-classical model design, and evaluation.

Workflow

The workflows of both the Quantum Multilayer Perceptron (QMLP) and the Quantum Convolutional Neural Network (QCNN) begin with malware datasets that are normalized and reduced using Principal Component Analysis (PCA) to align with the input size required by quantum circuits. Features are then embedded into quantum states using angle embedding. In the QMLP, the quantum circuit applies trainable rotation gates followed by qubit entanglement using CRX gates and data re-uploading to increase expressivity. In the QCNN, similar trainable rotations and CRX-based entanglement are followed by pooling operations to extract hierarchical features across layers. Both models measure qubit states using the Pauli-Z observable, and the resulting outputs are passed to classical neural network layers for final malware classification.

Circuit Design

Quantum circuit architectures used in QMLP and QCNN models.

Acknowledgments

Jesus Lopez, Viviana Cadena, and Saeefa Rubaiyet Nowmi are funded by the Department of Computer Science at The University of Texas at El Paso (UTEP).