Source-Agnostic Gravitational-Wave Detection with Transformers

This report presents work undertaken as part of the CERN Summer Student Programme 2022. The study investigates the use of unsupervised deep-learning networks for the identification of gravitational wave signals, which would widen the reach of current detection methods and offer the possibility of source-agnostic gravitational wave detection. In particular, performance of the attention-based Transformer architecture in an autoencoder configuration is studied, and compared to a previously developed recurrent LSTM autoencoder. Once trained on detector data, an autoencoder with a Transformer encoder and a convolutional decoder is used to identify gravitational wave signals as anomalies. The model is found to perform better than the LSTM model, even though it exhibits drawbacks in its reconstruction accuracy, the resolution of which would further improve its performance. A transformer-based model, which is known to perform efficiently on data consisting of long sequences, is found to be promising to detect anomalous signals, and therefore could be used in addition to the current detection methods to increase sensitivity to signals from unknown exotic sources.