Territory holders and non-territory holders in Ayu fish coexist only in the population growth process due to hysteresis

Ayu fish form feeding territories during a non-breeding (growing) season. When the density of the fish increases, phases gradually change. In the early growing season, all fish can hold territories at low density. Once all territory sites are occupied, newcomers become floaters. As the density further increases, territory holders have to spend much more time in defending their own territory and lose the time to feed on algae. Eventually, all fish give up their own territories and then form a school. In contrast, when the density decreases, territories are directly reformed from the school. In short, ayu fish exhibit a different transition, called hysteresis, where the two transitions occur widely-apart from each other. The dynamics of this intrinsic phenomena has not been demonstrated in previous studies. We develop a rate equation to describe the population dynamics within territorial competition. Our model successfully reproduces territorial hysteresis and indicates that territory holders and floaters can coexist only in the process of population growth. Moreover, we also find that the two critical densities of territorial hysteresis are conspicuously different from each other when the increase of the density of floaters sharply influences (step-function-like) the territories.