Evaluating the power of a recent method for comparing two circular distributions: an alternative to the Watson U(2) test

Some data are collected on circular (rather than linear) scales. Often researchers are interested in comparing two samples of such circular data to test the hypothesis that they came from the same underlying population. Recently, we compared 18 statistical approaches to testing such a hypothesis, and recommended two as particularly effective. A very recent publication introduced a novel statistical approach that was claimed to outperform the methods that we had indicated were highest performing. However, the evidence base for this claim was limited. Here we perform simulation studies to offer a more detailed comparison of the new “Angular Randomisation Test” (ART) with existing tests. We expand previous evaluations in two ways: exploring small and medium sized samples, and exploring a range of different shapes for the underlying distribution(s). We find that the ART controls type I error rates at the nominal level. The ART had greater power than established methods in detecting a difference in underlying distribution caused by a shift around the circle. Its performance advantage in this case was strongest when samples where small and unbalanced in size. When the difference between underlying unimodal distributions was in shape rather than central tendency, then the ART was at least as good (and sometimes considerably more powerful) than the established methods, except when distributions samples were small and uneven in size, and the smaller sample came from a more concentrated underlying distribution. In such cases its power could be markedly inferior to established alternatives. The ART was also inferior to alternatives in dealing with axially distributed data. We conclude that under widely-encountered circumstances the ART test can be recommended for its simplicity of implementation, but researchers should be aware of situations where it cannot be recommended.