Optimising functions with Quantum Annealers

No Thumbnail Available

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis

Date

2024-03-11

Department

Major/Subject

Machine Learning, Data Science and Artificial Intelligence

Mcode

SCI3044

Degree programme

Master’s Programme in Computer, Communication and Information Sciences

Language

en

Pages

46+2

Series

Abstract

We present, QP-SBGD, a novel layer-wise stochastic optimizer tailored towards training neural networks with binary weights, known as binary neural networks (BNNs), on quantum hardware. BNNs reduce the computational requirements and energy consumption of deep learning models without compromising accuracy. However, training them in practice remains to be an open challenge. Most known BNN-optimizers either rely on projected updates or binarize weights post-training. Instead, QP-SBGD approximately projects the gradient onto binary variables, by solving a quadratic constrained binary optimization. Under empirically validated √ assumptions, we show that this update rule converges with a rate of O(1/ T). Moreover, we show how the NP-hard projection can be effectively executed on an adiabatic quantum annealer, harnessing recent advancements in quantum computation. We also introduce a binary projected version of this update rule and prove that if a fixed point exists in the binary variable space, the modified updates will converge to it. Last but not least, our algorithm is implemented layer-wise, making it suitable to train larger networks on resource-limited quantum hardware. Through extensive evaluations, we show that QP-SBGD outperforms or is on par with competitive and well-established baselines such as BinaryConnect, signSGD and ProxQuant when optimizing the Rosenbrock function and training various neural network setups.

Description

Supervisor

Kannala, Juho

Thesis advisor

Birdal, Tolga
Golyanik, Vladislav

Keywords

optimisation, computer vision, quantum computing, machine learning

Other note

Citation