Studies of new Higgs boson interactions through nonresonant $HH$ production in the $b\bar{b}\gamma\gamma$ final state in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector

A search for nonresonant Higgs boson pair production in the $b\bar{b}\gamma\gamma$ final state is performed using 140 fb$^{-1}$ of proton-proton collisions at a centre-of-mass energy of 13 TeV recorded by the ATLAS detector at the CERN Large Hadron Collider. The analysis strategy is optimised to probe anomalous values of the Higgs ($H$) boson self-coupling modifier $\kappa_\lambda$ and of the quartic $HHVV$ ($V=W,Z$) coupling modifier $\kappa_{2V}$. No significant excess above the expected background from Standard Model processes is observed. An observed upper limit $\mu_{HH}<4.0$ is set at 95% confidence level on the Higgs boson pair production cross-section normalised to its Standard Model prediction. The 95% confidence intervals for the coupling modifiers are $-1.4<\kappa_\lambda<6.9$ and $-0.5<\kappa_{2V}<2.7$, assuming all other Higgs boson couplings except those under study are fixed to the Standard Model predictions. The results are also interpreted in the Standard Model effective field theory and Higgs effective field theory frameworks in terms of constraints on the couplings of anomalous Higgs boson (self-)interactions. This analysis supersedes and expands upon the previous nonresonant ATLAS results in this final state based on the full Run 2 dataset.