One of the first steps in the analysis of genetic data, and a principal mission of biology, is to describe and categorize natural variation. A continuous pattern of differentiation (isolation by distance), where individuals found closer together in space are, on average, more genetically similar than individuals sampled farther apart, can confound attempts to categorize natural variation into groups. This is because current statistical methods for assigning individuals to discrete clusters cannot accommodate spatial patterns, and so are forced to use clusters to describe what is in fact continuous variation. As isolation by distance is common in nature, this is a substantial shortcoming of existing methods.
Results/Conclusions
In this study, we introduce a new statistical method for categorizing natural genetic variation - one that describes variation as a combination of continuous and discrete patterns. Using simulations and empirical data applications, we demonstrate that this method works well and can capture patterns in population genomic data without resorting to splitting populations where they can be described by continuous patterns of variation.