Law of iterated logarithm and model selection consistency for generalized linear models with independent and dependent responses

doi:10.1007/s11464-021-0900-2

Front. Math. China

2021, Vol. 16

Issue (3) : 825-856 https://doi.org/10.1007/s11464-021-0900-2

RESEARCH ARTICLE

Law of iterated logarithm and model selection consistency for generalized linear models with independent and dependent responses

Xiaowei YANG¹, Shuang SONG^2,³, Huiming ZHANG^4,⁵(

)

¹. School of Mathematics and Statistics, Chaohu University, Chaohu 238024, China
². Center for Statistical Science, Tsinghua University, Beijing 100084, China
³. Department of Industrial Engineering, Tsinghua University, Beijing 100084, China
⁴. Department of Mathematics, University of Macau, Taipa Macau, China
⁵. UMacau Zhuhai Research Institute, Zhuhai 519000, China

Download: PDF(359 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

We study the law of the iterated logarithm (LIL) for the maximum likelihood estimation of the parameters (as a convex optimization problem) in the generalized linear models with independent or weakly dependent ( $ρ$ -mixing) responses under mild conditions. The LIL is useful to derive the asymptotic bounds for the discrepancy between the empirical process of the log-likelihood function and the true log-likelihood. The strong consistency of some penalized likelihood-based model selection criteria can be shown as an application of the LIL. Under some regularity conditions, the model selection criterion will be helpful to select the simplest correct model almost surely when the penalty term increases with the model dimension, and the penalty term has an order higher than O(log log n) but lower than O(n): Simulation studies are implemented to verify the selection consistency of Bayesian information criterion.

Keywords Generalized linear models (GLMs) weighted scores method non-naturallink function model selection consistency weakly dependent

Corresponding Author(s): Huiming ZHANG

Issue Date: 14 July 2021

Cite this article:

Xiaowei YANG,Shuang SONG,Huiming ZHANG. Law of iterated logarithm and model selection consistency for generalized linear models with independent and dependent responses[J]. Front. Math. China, 2021, 16(3): 825-856.

URL:

https://academic.hep.com.cn/fmc/EN/10.1007/s11464-021-0900-2
https://academic.hep.com.cn/fmc/EN/Y2021/V16/I3/825

1	M Y Ai, F Wang, J Yu, H M Zhang. Optimal subsampling for large-scale quantile regression. J Complexity, 2021, 62: 101512 https://doi.org/10.1016/j.jco.2020.101512
2	M Y Ai, J Yu, H M Zhang, H Y Wang. Optimal subsampling algorithms for big data regressions. Statist Sinica, 2021, 31(2): 749–772 https://doi.org/10.5705/ss.202018.0439
3	H Akaike. Information theory and an extension of the maximum likelihood principle. In: Second International Symposium on Information Theory. 1973, 267–281
4	D Bosq. Nonparametric Statistics for Stochastic Processes: Estimation and Prediction. Lect Notes Stat, Vol 110. Berlin: Springer, 1998 https://doi.org/10.1007/978-1-4612-1718-3
5	L D Brown. Fundamentals of Statistical Exponential Families: with Applications in Statistical Decision Theory. Inst Math Stat Lecture Notes-Monogr Ser, Vol 9. Hayward: Inst Math Stat, 1986
6	X R Chen. Quasi Likelihood Method for Generalized Linear Model. Hefei: Press of University of Science and Technology of China, 2011 (in Chinese)
7	C Czado, A Munk. Noncanonical links in generalized linear models when is the effort justified? J Statist Plann Inference, 2000, 87(2): 317–345 https://doi.org/10.1016/S0378-3758(99)00195-0
8	B Efron, T C Hastie. Computer Age Statistical Inference: Algorithms, Evidence, and Data Science. Cambridge: Cambridge Univ Press, 2016 https://doi.org/10.1017/CBO9781316576533
9	L Fahrmeir, H Kaufmann. Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models. Ann Statist, 1985, 13(1): 342–368 https://doi.org/10.1214/aos/1176346597
10	L Fahrmeir, G Tutz. Multivariate Statistical Modelling Based on Generalized Linear Models. 2nd ed. New York: Springer, 2001 https://doi.org/10.1007/978-1-4757-3454-6
11	J Q Fan, L Qi, X Tong. Penalized least squares estimation with weakly dependent data. Sci China Math, 2016, 59(12): 2335–2354 https://doi.org/10.1007/s11425-016-0098-x
12	X Z Fang. Laws of the iterated logarithm for maximum likelihood estimates of parameter vectors in nonhomogeneous Poisson processes. Acta Sci Natur Univ Pekinensis, 1998, 34(5): 563–573
13	B Hansen. Econometrics. Version: Jan 2018. 2018
14	X M He, G Wang. Law of the iterated logarithm and invariance principle for M-estimators. Proc Amer Math Soc, 1995, 123(2): 563–573 https://doi.org/10.2307/2160915
15	Y D Kim , J J Jeon. Consistent model selection criteria for quadratically supported risks. Ann Statist, 2016, 44(6): 2467–2496 https://doi.org/10.1214/15-AOS1413
16	M Kroll. Non-parametric Poisson regression from independent and weakly dependent observations by model selection. J Statist Plann Inference, 2019, 199: 249–270 https://doi.org/10.1016/j.jspi.2018.07.003
17	T L Lai, C Z Wei. A law of the iterated logarithm for double arrays of independent random variables with applications to regression and time series models. Ann Probab, 1982, 10(2): 320–335 https://doi.org/10.1214/aop/1176993860
18	Z Y Lin, C R Lu. Limit Theory for Mixing Dependent Random Variables. Mathematics and Its Applications. Beijing/Dordrecht: Science Press/Kluwer Academic Publishers,1997
19	M W Mahoney, J C Duchi , A C Gilbert . The Mathematics of Data. Providence: Amer Math Soc, 2018 https://doi.org/10.1090/pcms/025
20	M Markatou , A Basu, B G Lindsay. Weighted likelihood equations with bootstrap root search. J Amer Statist Assoc, 1998, 93(442): 740–750 https://doi.org/10.1080/01621459.1998.10473726
21	P McCullagh, J A Nelder. Generalized Linear Models. 2nd ed. London: Chapman and Hall, 1989 https://doi.org/10.1007/978-1-4899-3242-6
22	Y Miao , G Y Yang. The loglog law for LS estimator in simple linear EV regression models. Statistics, 2011, 45(2): 155–162 https://doi.org/10.1080/02331880903450576
23	J A Nelder , R W M Wedderburn. Generalized linear models. J R Statist Soc Ser A, 1972, 135(3): 370–384 https://doi.org/10.2307/2344614
24	G Q Qian , Y H Wu. Strong limit theorems on model selection in generalized linear regression with binomial responses. Statist Sinica, 2006, 16(4): 1335–1365
25	C R Rao , Y H Wu. A strongly consistent procedure for model selection in a regression problem. Biometrika, 1989, 76(2): 369–374 https://doi.org/10.1093/biomet/76.2.369
26	C R Rao , L C Zhao. Linear representation of M-estimates in linear models. Canad J Statist, 1992, 20(4): 359–368 https://doi.org/10.2307/3315607
27	P Rigollet . Kullback-Leibler aggregation and misspecified generalized linear models. Ann Statist, 2012, 40(2): 639–665 https://doi.org/10.1214/11-AOS961
28	J Rissanen . Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific, 1989
29	G Schwarz . Estimating the dimension of a model. Ann Statist, 1978, 6(2): 461–464 https://doi.org/10.1214/aos/1176344136
30	J Shao. Mathematical Statistics. 2nd ed. New York: Springer, 2003 https://doi.org/10.1007/b97553
31	W F Stout . Almost Sure Convergence. New York: Academic Press, 1974
32	G Tutz . Regression for Categorical Data. Cambridge: Cambridge Univ Press, 2011
33	A W van der Vaart. Asymptotic Statistics. Cambridge: Cambridge Univ Press, 1998
34	Y Wu , M M Zen. A strongly consistent information criterion for linear model selection based on M-estimation. Probab Theory Related Fields, 1999, 113(4): 599–625 https://doi.org/10.1007/s004400050219
35	C C Yin, L C Zhao, C D Wei. Asymptotic normality and strong consistency of maximum quasi-likelihood estimates in generalized linear models. Sci China Ser A, 2006, 49(2): 145–157 https://doi.org/10.1007/s11425-004-5169-x
36	H Zhang , J Jia. Elastic-net regularized high-dimensional negative binomial regression: consistency and weak signals detection. Statist Sinica (to appear)
37	H M Zhang , K Tan, B Li . COM-negative binomial distribution: modeling overdispersion and ultrahigh zero-inated count data. Front Math China, 2018, 13(4): 967–998 https://doi.org/10.1007/s11464-018-0714-z

[1]	Tianqing LIU,Zhidong BAI,Baoxue ZHANG. Weighted estimating equation: modified GEE in longitudinal data analysis[J]. Front. Math. China, 2014, 9(2): 329-353.

Viewed

Full text

Abstract

Cited

Shared

Discussed