MBE Advance Access published online on March 24, 2009
Molecular Biology and Evolution, doi:10.1093/molbev/msp059
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Research Article |
Inferring population mutation rate and sequencing error rate using the SNP frequency spectrum in a sample of DNA sequences
Human Genetics Center, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX 77030
Corresponding Authors: Yun-Xin Fu, Human Genetics Center, The University of Texas at Houston, PO Box 20186, Houston, Texas 77225 E-mail: Yunxin.Fu{at}uth.tmc.edu Phone: 713-500-9813
Received for publication January 30, 2009. Revision received March 15, 2009. Accepted for publication March 18, 2009.
One challenge of analyzing samples of DNA sequences is to account for the non-negligible polymorphisms produced by error when the sequencing error rate is high or the sample size is large. Specifically, those artificial sequence variations will bias the observed SNP frequency spectrum, which in turn may further bias the estimators of the population mutation rate
= 4Nµ for diploids. In this paper, we propose a new approach based on the generalized least squares (GLS) method to estimate
, given a SNP frequency spectrum in a random sample of DNA sequences from a population. With this approach, error rate e can be either known or unknown. In the latter case
can be estimated given an estimation of
. Using coalescent simulation, we compared our estimators with other estimators of
. The results showed the GLS estimators are more efficient than other
estimators with error, and the estimation of
is usable in practice when the
per bp is small. We demonstrate the application of the estimators with 10kb noncoding region sequence sampled from a human population and provide suggestions for choosing
estimators with error
Key Words: coalescent theory sequencing error mutation rate SNP frequency spectrum generalized least squares