A Comparative Study of Classical and Quantum Transformer Models and Their Applications

Abstract

Transformers are a neural network architecture that enable the utilisation of a larger context frame than traditional neural networks when training deep learning models. With the advent of the ongoing decade, transformers have enabled the application of large language models, improved computer vision, generative models and other large-scale artificial intelligence systems.

This thesis investigates the potential of quantum transformers to implement deep learning tasks focusing on the Quixer model, developed by Quantinuum, by comparing a classically simulated version of Quixer to classical transformers. The thesis is motivated by the need to optimize transformer components for large-scale applications, to address the quadratic complexity arising from the self attention mechanism. The results of the thesis indicate that Quixer performs in line with the classical baseline as published in the paper by Quantinuum when reproduced with the same dataset, and model performance follows the trend for another dataset of twice the size. Hence, providing a proof of concept quantum transformers can be considered an effective method for developing large-scale models in the future with the eventual improvement of quantum hardware.