Statistical Inference for High Variance Gaussian Mixture Models
Anya Katsevich, New York University
We derive an asymptotic expansion for the log likelihood of spherical Gaussian mixture models (GMMs) in the low signal-to-noise (SNR) regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the Method of Moments (MoM) and likelihood-optimizing algorithms such as Expectation Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone toward the analysis of EM and maximum likelihood estimation in a wide range of models. It also implies that the computationally efficient MoM performs just as well as the much more computationally intensive EM method in the low SNR regime. We demonstrate this numerically by applying EM and MoM to a low SNR GMM using a large number of samples. We implemented a GPU parallelized version of EM to make the computation tractable. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers.
Abstract Author(s): Anya Katsevich, Afonso Bandeira