Spatial autoregressive (SAR) models with varying coefficients are useful for capturing heterogeneous effects of the impacts of covariates as well as spatial interaction in empirical studies, and a wide range of popular models can be seen as its special cases, such as linear SAR models. In this study, we will propose a unified model selection method for the SAR model with varying coefficients to achieve two targets simultaneously: (1) variable selection (eliminate irrelevant covariates), and (2) identification of the covariates with constant effect among the relevant covariates. To do so, we follow the idea of group LASSO to incorporate two penalty functions to simultaneously do model selection and estimation. Monte Carlo experiments show that the proposed method performs well in finite samples. Finally, we illustrate the method with an application to the housing data of Chinese cities.