Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework

doi:10.1007/s11704-016-5552-0

Front. Comput. Sci.

2017, Vol. 11

Issue (3) : 362-391 https://doi.org/10.1007/s11704-016-5552-0

REVIEW ARTICLE

Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework

Maoguo GONG(

), Xiangming JIANG, Hao LI

Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, China

Download: PDF(722 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Ill-posed problems are widely existed in signal processing. In this paper, we review popular regularization models such as truncated singular value decomposition regularization, iterative regularization, variational regularization. Meanwhile, we also retrospect popular optimization approaches and regularization parameter choice methods. In fact, the regularization problem is inherently a multiobjective problem. The traditional methods usually combine the fidelity term and the regularization term into a singleobjective with regularization parameters, which are difficult to tune. Therefore, we propose a multi-objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity. In this framework, the fidelity term and regularization term are optimized simultaneously to gain more insights into the ill-posed problems. A case study on signal recovery shows the effectiveness of the multi-objective framework for ill-posed problems.

Keywords ill-posed problem regularization multiobjective optimization evolutionary algorithm signal processing

Corresponding Author(s): Maoguo GONG

Just Accepted Date: 23 June 2016 Online First Date: 19 September 2016 Issue Date: 25 May 2017

Cite this article:

Maoguo GONG,Xiangming JIANG,Hao LI. Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework[J]. Front. Comput. Sci., 2017, 11(3): 362-391.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-016-5552-0
https://academic.hep.com.cn/fcs/EN/Y2017/V11/I3/362

1	HadamardJ. Sur les Problemes aux Derivees Partielles et Leur Signification Physique. Princeton University Bulletin, 1902, 13: 49–52
2	KabanikhinS I. Inverse and Ill-Posed Problems: Theory and Applications. Berlin: Water De Gruyter, 2011 https://doi.org/10.1515/9783110224016
3	ZhangB Y, XuD H, LiuT W. Stabilized algorithms for ill-posed problems in signal processing. In: Proceedings of the IEEE International Conferences on Info-tech and Info-net. 2001, 1: 375–380 https://doi.org/10.1109/ICII.2001.982776
4	ScherzerO. Handbook of Mathematical Methods in Imaging. Springer Science & Business Media, 2011 https://doi.org/10.1007/978-0-387-92920-0
5	GroetschC W. Inverse problems in the mathematical sciences. Mathematics of Computation, 1993, 63(5): 799–811 https://doi.org/10.1007/978-3-322-99202-4
6	RudinL I, OsherS, FatemiE. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 1992, 60(1): 259–268 https://doi.org/10.1016/0167-2789(92)90242-F
7	TikhonovA N. Solution of incorrectly formulated problems and the regularization method. Soviet Math, 1963, 4: 1035–1038
8	TikhonovA N, Arsenin V I. Solutions of Ill-posed Problems. Washington, DC: V. H. Winston & Sons, 1977
9	LandweberL. An iteration formula for Fredholm integral equations of the first kind. American Journal of Mathematics, 1951, 73(3): 615–624 https://doi.org/10.2307/2372313
10	HestenesM R, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409–436 https://doi.org/10.6028/jres.049.044
11	VogelC R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002, 23 https://doi.org/10.1137/1.9780898717570
12	HansenP C. The truncated SVD as a method for regularization. Bit Numerical Mathematics, 1987, 27(4): 534–553 https://doi.org/10.1007/BF01937276
13	HonerkampJ, WeeseJ. Tikhonovs regularization method for ill-posed problems. Continuum Mechanics and Thermodynamics, 1990,2(1): 17–30 https://doi.org/10.1007/BF01170953
14	ZhangX Q, BurgerM, BressonX, Osher S. Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 2010, 3(3): 253–276 https://doi.org/10.1137/090746379
15	DebK. Multi-Objective Optimization Using Evolutionary Algorithms. New York: John Wiley & Sons, 2001, 16
16	FonsecaC M, Fleming P J. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 1995, 3(1): 1–16 https://doi.org/10.1162/evco.1995.3.1.1
17	CoelloC A C, Van Veldhuizen D A, LamontG B . Evolutionary Algorithms for Solving Multi-objective Problems. New York: Kluwer Academic, 2002 https://doi.org/10.1007/978-1-4757-5184-0
18	TanK C, KhorE F, LeeT H. Multiobjective Evolutionary Algorithms and Applications. Springer Science & Business Media, 2005
19	KnowlesJ, CorneD, DebK. Multiobjective Problem Solving from Nature: from Concepts to Applications. Springer Science & Business Media, 2008 https://doi.org/10.1007/978-3-540-72964-8
20	RaquelC, YaoX. Dynamic multi-objective optimization: a survey of the state-of-the-art. In: YangS X , YaoX, eds. Evolutionary Computation for Dynamic Optimization Problems. Springer Berlin Heidelberg, 2013, 85–106 https://doi.org/10.1007/978-3-642-38416-5_4
21	LückenC V, Barán B, BrizuelaC . A survey on multi-objective evolutionary algorithms for many-objective problems. Computational Optimization and Applications, 2014, 58(3): 707–756 https://doi.org/10.1007/s10589-014-9644-1
22	HwangC L, MasudA S M.Multiple Objective Decision Making- Methods and Applications. Springer Science & Business Media, 1979, 164 https://doi.org/10.1007/978-3-642-45511-7
23	GirosiF, JonesM B, PoggioT. Regularization theory and neural networks architectures. Neural Computation, 1995, 7(2): 219–269 https://doi.org/10.1162/neco.1995.7.2.219
24	BelgeM, KilmerM E, MillerE L. Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Problems, 2002, 18(4): 1161 https://doi.org/10.1088/0266-5611/18/4/314
25	HansenP C. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. American Mathematical Monthly, 1997, 4(5): 491
26	ErikssonP, Jiménez C, BuehlerS A . Qpack, a general tool for instrument simulation and retrieval work.Journal of Quantitative Spectroscopy and Radiative Transfer, 2005, 91(1): 47–64 https://doi.org/10.1016/j.jqsrt.2004.05.050
27	GiustiE. Minimal Surfaces and Functions of Bounded Variation. Springer Science & Business Media, 1984, 80 https://doi.org/10.1007/978-1-4684-9486-0
28	CattéF, CollT.Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 1992, 29(1): 182–193 https://doi.org/10.1137/0729012
29	BjörckA. Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, 1996 https://doi.org/10.1137/1.9781611971484
30	GroetschC W. The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman Advanced Publishing Program, 1984
31	HansonR J. A numerical method for solving Fredholm integral equationstific and Statistical Computing, 1992, 13(5): 1142–1150
32	StewartG W. Rank degeneracy. SIAM Journal on Scientific and Statistical Computing, 1984, 5(2): 403–413 https://doi.org/10.1137/0905030
33	HansenP C, SekiiT, ShibahashiH . The modified truncated SVD method for regularization in general form. SIAM Journal on Sciendependent Component Analysis and Blind Source Separation. 2006, 206–213
34	Van LoanC F. Generalizing the singular value decomposition. SIAM Journal on Numerical Analysis, 1976, 13(1): 76–83 https://doi.org/10.1137/0713009
35	HansenP C.Regularization, GSVD and truncated GSVD. BIT Numerical Mathematics, 1989, 29(3): 491–504 https://doi.org/10.1007/BF02219234
36	PaigeC C. Computing the generalized singular value decomposition. SIAM Journal on Scientific and Statistical Computing, 1986, 7(4): 1126–1146 https://doi.org/10.1137/0907077
37	MorigiS, Reichel L, SgallariF . A truncated projected SVD method for linear discrete ill-posed problems. Numerical Algorithms, 2006, 43(3): 197–213 https://doi.org/10.1007/s11075-006-9053-3
38	FernandoK V, Hammarling S. A product induced singular value decomposition (ΠSVD) for two matrices and balanced realization. In: Proceedings of SIAM Conference on Linear Algebra in Signals, Systems and Control. 1988, 128–140
39	ZhaH Y. The restricted singular value decomposition of matrix triplets. SIAM Journal on Matrix Analysis and Applications, 1991, 12(1): 172–194 https://doi.org/10.1137/0612014
40	De MoorB, GolubG H. The restricted singular value decomposition: properties and applications. SIAM Journal on Matrix Analysis and Applications, 1991, 12(3): 401–425 https://doi.org/10.1137/0612029
41	De MoorB, ZhaH Y. A tree of generalizations of the ordinary singular value decomposition. Linear Algebra and Its Applications, 1991, 147: 469–500 https://doi.org/10.1016/0024-3795(91)90243-P
42	De MoorB. Generalizations of the OSVD: structure, properties and applications. In: VaccaroR J, ed.SVD & Signal Processing, II: Algorithms, Analysis & Applications. 1991, 83–98
43	NoscheseS, Reichel L. A modified TSVD method for discrete illposed problems. Numerical Linear Algebra with Applications, (in press)
44	DykesL, Noschese S, ReichelL . Rescaling the GSVD with application to ill-posed problems. Numerical Algorithms, 2015, 68(3): 531–545 https://doi.org/10.1007/s11075-014-9859-3
45	EdoL, FrancoW, MartinssonP G , RokhlinV, TygertM. Randomized algorithms for the low-rank approximation of matrices. Proceedings of the National Academy of Sciences, 2007, 104(51): 20167–20172 https://doi.org/10.1073/pnas.0709640104
46	WoolfeF, Liberty E, RokhlinV , TygertM. A fast randomized algorithm for the approximation of matrices. Applied & Computational Harmonic Analysis, 2008, 25(3): 335–366 https://doi.org/10.1016/j.acha.2007.12.002
47	SifuentesJ, Gimbutas Z, GreengardL . Randomized methods for rankdeficient linear systems. Electronic Transactions on Numerical Analysis, 2015, 44: 177–188
48	LiuY G, LeiY J, LiC G, Xu W Z, PuY F . A random algorithm for low-rank decomposition of large-scale matrices with missing entries. IEEE Transactions on Image Processing, 2015, 24(11): 4502–4511 https://doi.org/10.1109/TIP.2015.2458176
49	SekiiT. Two-dimensional inversion for solar internal rotation. Publications of the Astronomical Society of Japan, 1991, 43: 381–411
50	ScalesJ A. Uncertainties in seismic inverse calculations. In: Jacobsen B H, MosegaardK , SibaniP, eds. Inverse Methods. Berlin: Springer- Verlag, 1996, 79–97 https://doi.org/10.1007/BFb0011766
51	LawlessJ F, WangP. A simulation study of ridge and other regression estimators. Communications in Statistics-Theory and Methods, 1976, 5(4): 307–323 https://doi.org/10.1080/03610927608827353
52	DempsterA P, Schatzoff M, WermuthN . A simulation study of alternatives to ordinary least squares. Journal of the American Statistical Association, 1977, 72(357): 77–91 https://doi.org/10.1080/01621459.1977.10479910
53	HansenP C, O’Leary D P. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 1993, 14(6): 1487–1503 https://doi.org/10.1137/0914086
54	HansenP C. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Review, 1992, 34(4): 561–580 https://doi.org/10.1137/1034115
55	XuP L. Truncated SVD methods for discrete linear ill-posed problems. Geophysical Journal International, 1998, 135(2): 505–514 https://doi.org/10.1046/j.1365-246X.1998.00652.x
56	WuZ M, BianS F, XiangC B, Tong Y D. A new method for TSVD regularization truncated parameter selection. Mathematical Problems in Engineering, 2013 https://doi.org/10.1155/2013/161834
57	ChiccoD, Masseroli M. A discrete optimization approach for SVD best truncation choice based on ROC curves. In: Proceedings of the 13th IEEE International Conference on Bioinformatics and Bioengineering. 2013: 1–4 https://doi.org/10.1109/BIBE.2013.6701705
58	GolubG H, HeathM, WahbaG. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 1979, 21(2): 215–223 https://doi.org/10.1080/00401706.1979.10489751
59	JbilouK, Reichel L, SadokH . Vector extrapolation enhanced TSVD for linear discrete ill-posed problems. Numerical Algorithms, 2009, 51(2): 195–208 https://doi.org/10.1007/s11075-008-9229-0
60	BouhamidiA, JbilouK, ReichelL, Sadok H, WangZ . Vector extrapolation applied to truncated singular value decomposition and truncated iteration. Journal of Engineering Mathematics, 2015, 93(1): 99–112 https://doi.org/10.1007/s10665-013-9677-y
61	VogelC R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002 https://doi.org/10.1137/1.9780898717570
62	DoicuA, Trautmann T, SchreierF . Numerical Regularization for Atmospheric Inverse Problems. Springer Science & Business Media, 2010 https://doi.org/10.1007/978-3-642-05439-6
63	BakushinskyA B, Goncharsky A V. Iterative Methods for the Solution of Incorrect Problems. Moscow: Nauka, 1989
64	RiederA. Keine Probleme mit Inversen Problemen: Eine Einführung in ihre stabile Lösung. Berlin: Springer-Verlag, 2013
65	NemirovskiyA S, PolyakB T.Iterative methods for solving linear illposed problems under precise information. Engineering Cybernetics, 1984, 22(4): 50–56
66	BrakhageH. On ill-posed problems and the method of conjugate gradients. Inverse and Ill-posed Problems, 1987, 4: 165–175 https://doi.org/10.1016/B978-0-12-239040-1.50014-4
67	HankeM. Accelerated Landweber iterations for the solution of illposed equations. Numerische Mathematik, 1991, 60(1): 341–373 https://doi.org/10.1007/BF01385727
68	BarzilaiJ, Borwein J M. Two-point step size gradient methods. IMA Journal of Numerical Analysis, 1988, 8(1): 141–148 https://doi.org/10.1093/imanum/8.1.141
69	AxelssonO. Iterative Solution Methods. Cambridge:Cambridge University Press, 1996
70	Van der SluisA, Van der Vorst H A. The rate of convergence of conjugate gradients. Numerische Mathematik, 1986, 48(5): 543–560 https://doi.org/10.1007/BF01389450
71	ScalesJ A, Gersztenkorn A. Robust methods in inverse theory. Inverse Problems, 1988, 4(4): 1071–1091 https://doi.org/10.1088/0266-5611/4/4/010
72	BjörckÅ, Eldén L. Methods in numerical algebra for ill-posed problems. Technical Report LiTH-MAT-R-33-1979. 1979
73	TrefethenL N, BauD. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997 https://doi.org/10.1137/1.9780898719574
74	CalvettiD, LewisB, ReichelL. On the regularizing properties of the GMRES method. Numerische Mathematik, 2002, 91(4): 605–625 https://doi.org/10.1007/s002110100339
75	CalvettiD, LewisB, ReichelL. Alternating Krylov subspace image restoration methods. Journal of Computational and Applied Mathematics, 2012, 236(8): 2049–2062 https://doi.org/10.1016/j.cam.2011.09.030
76	BrianziP, FavatiP, MenchiO, Romani F. A framework for studying the regularizing properties of Krylov subspace methods.Inverse Problems, 2006, 22(3): 1007–1021 https://doi.org/10.1088/0266-5611/22/3/017
77	SonneveldP, Van Gijzen M B. IDR(s): a family of simple and fast algorithms for solving large nonsymmetric systems of linear equations. SIAM Journal on Scientific Computing, 2008, 31(2): 1035–1062 https://doi.org/10.1137/070685804
78	FongD C L, Saunders M. LSMR: an iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing, 2011, 33(5): 2950–2971 https://doi.org/10.1137/10079687X
79	ZhaoC, HuangT Z, ZhaoX L, Deng L J. Two new efficient iterative regularization methods for image restoration problems. Abstract & Applied Analysis, 2013 https://doi.org/10.1155/2013/129652
80	PerezA, Gonzalez R C. An iterative thresholding algorithm for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1987, 9(6): 742–751 https://doi.org/10.1109/TPAMI.1987.4767981
81	BeckA, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202 https://doi.org/10.1137/080716542
82	Bioucas-DiasJ M, Figueiredo M A T. Two-step algorithms for linear inverse problems with non-quadratic regularization. In: Proceedings of the IEEE International Conference on Image Processing. 2007, 105–108 https://doi.org/10.1109/icip.2007.4378902
83	Bioucas-DiasJ M, Figueiredo M A T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing. 2007, 16(12): 2992–3004 https://doi.org/10.1109/TIP.2007.909319
84	BayramI, Selesnick I W. A subband adaptive iterative shrinkage/ thresholding algorithm. IEEE Transactions on Signal Processing, 2010, 58(3): 1131–1143 https://doi.org/10.1109/TSP.2009.2036064
85	YamagishiM, YamadaI. Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize. Inverse Problems, 2011, 27(10): 105008–105022 https://doi.org/10.1088/0266-5611/27/10/105008
86	BhottoM Z A, AhmadM O, SwamyM N S. An improved fast iterative shrinkage thresholding algorithm for image deblurring. SIAM Journal on Imaging Sciences, 2015, 8(3): 1640–1657 https://doi.org/10.1137/140970537
87	ZhangY D, DongZ C, PhillipsP, Wang S H, JiG L , YangJ Q. Exponential wavelet iterative shrinkage thresholding algorithm for compressed sensing magnetic resonance imaging. Information Sciences, 2015, 322: 115–132 https://doi.org/10.1016/j.ins.2015.06.017
88	ZhangY D, WangS H, JiG L, Dong Z C. Exponential wavelet iterative shrinkage thresholding algorithm with random shift for compressed sensing magnetic resonance imaging. IEEJ Transactions on Electrical and Electronic Engineering, 2015, 10(1): 116–117 https://doi.org/10.1002/tee.22059
89	WuG M, LuoS Q. Adaptive fixed-point iterative shrinkage/ thresholding algorithm for MR imaging reconstruction using compressed sensing. Magnetic Resonance Imaging, 2014, 32(4): 372–378 https://doi.org/10.1016/j.mri.2013.12.009
90	FangE X, WangJ J, HuD F, Zhang J Y, ZouW , ZhouY. Adaptive monotone fast iterative shrinkage thresholding algorithm for fluorescence molecular tomography. IET Science Measurement Technology, 2015, 9(5): 587–595 https://doi.org/10.1049/iet-smt.2014.0030
91	ZuoW M, MengD Y, ZhangL, Feng X C, ZhangD . A generalized iterated shrinkage algorithm for non-convex sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 217–224 https://doi.org/10.1109/iccv.2013.34
92	KrishnanD, FergusR. Fast image deconvolution using hyperlaplacian priors. In: BengioY , SchuurmansD, Lafferty J D, et al., eds. Advances in Neural Information Processing Systems 22. 2009, 1033–1041
93	ChartrandR, YinW. Iteratively reweighted algorithms for compressive sensing. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3869–3872 https://doi.org/10.1109/icassp.2008.4518498
94	SheY Y. An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors. Computational Statistics & Data Analysis, 2012, 56(10): 2976–2990 https://doi.org/10.1016/j.csda.2011.11.013
95	GongP H, ZhangC S, LuZ S, Huang J Z, YeJ P . A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of International Conference on Machine Learning. 2013, 37–45
96	BrediesK, LorenzD A. Linear convergence of iterative softthresholding.Journal of Fourier Analysis and Applications, 2008, 14(5–6): 813–837 https://doi.org/10.1007/s00041-008-9041-1
97	KowalskiM. Thresholding rules and iterative shrinkage/thresholding algorithm: a convergence study. In: Proceedings of the IEEE International Conference on Image Processing. 2014, 4151–4155 https://doi.org/10.1109/icip.2014.7025843
98	ChambolleA, DossalC. On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. Journal of Optimization Theory & Applications, 2015, 166(3): 968–982 https://doi.org/10.1007/s10957-015-0746-4
99	JohnstoneP R, MoulinP. Local and global convergence of an inertial version of forward-backward splitting. Advances in Neural Information Processing Systems, 2014, 1970–1978
100	MorozovV A. On the solution of functional equations by the method of regularization. Soviet Mathematics Doklady, 1966, 7(11): 414–417
101	VainikkoG M. The discrepancy principle for a class of regularization methods. USSR Computational Mathematics and Mathematical Physics, 1982, 22(3): 1–19 https://doi.org/10.1016/0041-5553(82)90120-3
102	VainikkoG M. The critical level of discrepancy in regularization methods. USSR Computational Mathematics and Mathematical Physics, 1983, 23(6): 1–9 https://doi.org/10.1016/S0041-5553(83)80068-8
103	PlatoR. On the discrepancy principle for iterative and parametric methods to solve linear ill-posed equations. Numerische Mathematik, 1996, 75(1): 99–120 https://doi.org/10.1007/s002110050232
104	BorgesL S, Bazán F S V, CunhaM C C . Automatic stopping rule for iterative methods in discrete ill-posed problems. Computational & Applied Mathematics, 2015, 34(3): 1175–1197 https://doi.org/10.1007/s40314-014-0174-3
105	DziwokiG, Izydorczyk J. Stopping criteria analysis of the OMP algorithm for sparse channels estimation. In: Proceedings of the International Conference on Computer Networks. 2015, 250–259 https://doi.org/10.1007/978-3-319-19419-6_24
106	FavatiP, LottiG, MenchiO, Romani F.Stopping rules for iterative methods in nonnegatively constrained deconvolution. Applied Numerical Mathematics, 2014, 75: 154–166 https://doi.org/10.1016/j.apnum.2013.07.006
107	EnglH W, HankeM, NeubauerA. Regularization of Inverse Problems. Springer Science & Business Media, 1996 https://doi.org/10.1007/978-94-009-1740-8
108	AmsterP. Iterative Methods. Universitext, 2014, 53–82 https://doi.org/10.1007/978-1-4614-8893-4_3
109	WaseemM. On some iterative methods for solving system of nonlinear equations. Dissertation for the Doctoral Degree.Islamabad: COMSATS Institute of Information Technology, 2012
110	BurgerM, OsherS. A guide to the TV zoo. In: BurgerM, Mennucci A C G, OsherS , et al., eds. Level Set and PDE Based Reconstruction Methods in Imaging. Springer International Publishing, 2013, 1–70 https://doi.org/10.1007/978-3-319-01712-9_1
111	TikhonovA N. Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 1963, 4(1): 1624–1627
112	NikolovaM. Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Modeling & Simulation, 2005, 4(3): 960–991 https://doi.org/10.1137/040619582
113	BurgerM, OsherS. Convergence rates of convex variational regularization. Inverse Problems, 2004, 20(5): 1411–1421 https://doi.org/10.1088/0266-5611/20/5/005
114	HofmannB, Kaltenbacher B, PöschlC , ScherzerO. A convergence rates result for Tikhonov regularization in Banach spaces with nonsmooth operators. Inverse Problems, 2007, 23(3): 987–1010 https://doi.org/10.1088/0266-5611/23/3/009
115	ResmeritaE. Regularization of ill-posed problems in Banach spaces:convergence rates. Inverse Problems, 2005, 21(4): 1303–1314 https://doi.org/10.1088/0266-5611/21/4/007
116	ResmeritaE, Scherzer O. Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Problems, 2006, 22(3): 801–814 https://doi.org/10.1088/0266-5611/22/3/004
117	EnglH W. Discrepancy principles for Tikhonov regularization of illposed problems leading to optimal convergence rates. Journal of Optimization Theory and Applications, 1987, 52(2): 209–215 https://doi.org/10.1007/BF00941281
118	GfrererH. An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Mathematics of Computation, 1987, 49(180): 507–522 https://doi.org/10.1090/S0025-5718-1987-0906185-4
119	NattererF. Error bounds for Tikhonov regularization in Hilbert scales. Applicable Analysis, 1984, 18(1–2): 29–37 https://doi.org/10.1080/00036818408839508
120	NeubauerA. An a posteriori parameter choice for Tikhonov regularization in the presence of modeling error. Applied Numerical Mathematics, 1988, 4(6): 507–519 https://doi.org/10.1016/0168-9274(88)90013-X
121	EnglH W, Kunisch K, NeubauerA . Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Problems, 1989, 5(4): 523–540 https://doi.org/10.1088/0266-5611/5/4/007
122	ScherzerO, EnglH W, KunischK. Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM Journal on Numerical Analysis, 1993, 30(6): 1796–1838 https://doi.org/10.1137/0730091
123	VarahJ M. On the numerical solution of ill-conditioned linear systems with applications to ill-posed problems. SIAM Journal on Numerical Analysis, 1973, 10(2): 257–267 https://doi.org/10.1137/0710025
124	VinodH D, UllahA. Recent Advances in Regression Methods. Danbury: Marcel Dekker Incorporated, 1981
125	O’SullivanF. A statistical perspective on ill-posed inverse problems. Statistical Science, 1986, 1(4): 502–518 https://doi.org/10.1214/ss/1177013525
126	GrafarendE W, Schaffrin B. Ausgleichungsrechnung in linearen modellen. BI Wissenschaftsverlag Mannheim, 1993
127	RodgersC D. Inverse Methods for Atmospheric Sounding: Theory and Practice. Singapore: World Scientific, 2000 https://doi.org/10.1142/3171
128	CeccheriniS. Analytical determination of the regularization parameter in the retrieval of atmospheric vertical profiles. Optics Letters, 2005, 30(19): 2554–2556 https://doi.org/10.1364/OL.30.002554
129	MallowsC L. Some comments on Cp. Technometrics, 1973, 15(4): 661–675
130	RiceJ. Choice of smoothing parameter in deconvolution problems. Contemporary Mathematics, 1986, 59: 137–151 https://doi.org/10.1090/conm/059/10
131	HankeM, RausT. A general heuristic for choosing the regularization parameter in ill-posed problems. SIAM Journal on Scientific Computing, 1996, 17(4): 956–972 https://doi.org/10.1137/0917062
132	WuL M. A parameter choice method for Tikhonov regularization. Electronic Transactions on Numerical Analysis, 2003, 16: 107–128
133	GaoW, YuK P. A new method for determining the Tikhonov regularization parameter of load identification. In: Proceedings of the International Symposium on Precision Engineering Measurement and Instrumentation. 2015
134	ItoK, JinB, TakeuchiT. Multi-parameter Tikhonov regularizationan augmented approach. Chinese Annals of Mathematics, Series B, 2014, 35(03): 383–398 https://doi.org/10.1007/s11401-014-0835-y
135	JinB, LorenzD A. Heuristic parameter-choice rules for convex variational regularization based on error estimates. SIAM Journal on Numerical Analysis, 2010, 48(3): 1208–1229 https://doi.org/10.1137/100784369
136	PazosF, BhayaA. Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov optimizing control. Journal of Computational and Applied Mathematics, 2015, 279: 123–132 https://doi.org/10.1016/j.cam.2014.10.022
137	HämarikU, PalmR, RausT. A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level. Journal of Computational and Applied Mathematics, 2012, 236(8): 2146–2157 https://doi.org/10.1016/j.cam.2011.09.037
138	ReichelL, Rodriguez G. Old and new parameter choice rules for discrete ill-posed problems. Numerical Algorithms, 2013, 63(1): 65–87 https://doi.org/10.1007/s11075-012-9612-8
139	KryanevA V. An iterative method for solving incorrectly posed problems. USSR Computational Mathematics and Mathematical Physics, 1974, 14(1): 24–35 https://doi.org/10.1016/0041-5553(74)90133-5
140	KingJ T, Chillingworth D. Approximation of generalized inverses by iterated regularization. Numerical Functional Analysis & Optimization, 1979, 1(5): 499–513 https://doi.org/10.1080/01630567908816031
141	FakeevA G. A class of iterative processes for solving degenerate systems of linear algebraic equations. USSR Computational Mathematics and Mathematical Physics, 1981, 21(3): 15–22 https://doi.org/10.1016/0041-5553(81)90060-4
142	BrillM, SchockE. Iterative solution of ill-posed problems: a survey. In: Proceedings of the 4th International Mathematical Geophysics Seminar. 1987
143	HankeM, Groetsch C W. Nonstationary iterated Tikhonov regularization. Journal of Optimization Theory and Applications, 1998, 98(1): 37–53 https://doi.org/10.1023/A:1022680629327
144	LampeJ, Reichel L, VossH . Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra and Its Applications, 2012, 436(8): 2845–2865 https://doi.org/10.1016/j.laa.2011.07.019
145	ReichelL, YuX B. Tikhonov regularization via flexible Arnoldi reduction. BIT Numerical Mathematics, 2015, 55(4): 1145–1168 https://doi.org/10.1007/s10543-014-0542-9
146	HuangG, Reichel L, YinF . Projected nonstationary iterated Tikhonov regularization. BIT Numerical Mathematics, 2016, 56(2): 467–487 https://doi.org/10.1007/s10543-015-0568-7
147	AmbrosioL, FuscoN, PallaraD. Functions of Bounded Variation and Free Discontinuity Problems. Oxford: Oxford University Press, 2000
148	AcarR, VogelC R. Analysis of bounded variation penalty methods for ill-posed problems. Inverse Problems, 1997, 10(6): 1217–1229 https://doi.org/10.1088/0266-5611/10/6/003
149	HuntB R. The application of constrained least squares estimation to image restoration by digital computer. IEEE Transactions on Computers, 1973, 100(9): 805–812 https://doi.org/10.1109/TC.1973.5009169
150	DemomentG. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech and Signal Processing, 1989, 37(12): 2024–2036 https://doi.org/10.1109/29.45551
151	KatsaggelosA K. Iterative image restoration algorithms. Optical Engineering, 1989, 28(7): 735–748 https://doi.org/10.1117/12.7977030
152	KatsaggelosA K, Biemond J, SchaferR W , MersereauR M. A regularized iterative image restoration algorithm. IEEE Transactions on Signal Processing, 1991, 39(4): 914–929 https://doi.org/10.1109/78.80914
153	BabacanS D, MolinaR, KatsaggelosA K . Parameter estimation in TV image restoration using variational distribution approximation. IEEE Transactions on Image Processing, 2008, 17(3): 326–339 https://doi.org/10.1109/TIP.2007.916051
154	WenY W, ChanR H. Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Transactions on Image Processing, 2012, 21(4): 1770–1781 https://doi.org/10.1109/TIP.2011.2181401
155	ChenA, HuoB M, WenC W. Adaptive regularization for color image restoration using discrepancy principle. In: Proceedings of the IEEE International Conference on Signal processing, Comminications and Computing. 2013, 1–6 https://doi.org/10.1109/icspcc.2013.6663988
156	LinY, Wohlberg B, GuoH . UPRE method for total variation parameter selection. Signal Processing, 2010, 90(8): 2546–2551 https://doi.org/10.1016/j.sigpro.2010.02.025
157	SteinC M. Estimation of the mean of a multivariate normal distribution. Annals of Statistics, 1981, 9(6): 1135–1151 https://doi.org/10.1214/aos/1176345632
158	RamaniS, BluT, UnserM. Monte-Carlo SURE: a black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on Image Processing, 2008, 17(9): 1540–1554 https://doi.org/10.1109/TIP.2008.2001404
159	PalssonF, Sveinsson J R, UlfarssonM O , BenediktssonJ A. SAR image denoising using total variation based regularization with surebased optimization of regularization parameter. In: Proceedings of the IEEE International Conference on Geoscience and Remote Sensing Symposium. 2012, 2160–2163
160	LiaoH Y, LiF, NgM K. Selection of regularization parameter in total variation image restoration. Journal of the Optical Society of America A, 2009, 26(11): 2311–2320 https://doi.org/10.1364/JOSAA.26.002311
161	BertalmíoM, Caselles V, RougéB , SoléA. TV based image restoration with local constraints. Journal of Scientific Computing, 2003, 19(1–3): 95–122 https://doi.org/10.1023/A:1025391506181
162	AlmansaA, Ballester C, CasellesV , HaroG. A TV based restoration model with local constraints. Journal of Scientific Computing, 2008, 34(3): 209–236 https://doi.org/10.1007/s10915-007-9160-x
163	VogelC R, OmanM E. Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 1997, 17(1): 227–238 https://doi.org/10.1137/0917016
164	ChanT F, GolubG H, MuletP. A nonlinear primal-dual method for total variation-based image restoration. Lecture Notes in Control & Information Sciences, 1995, 20(6): 1964–1977
165	ChambolleA. An algorithm for total variation minimization and applications. Journal ofMathematical Imaging & Vision, 2004, 20(1–2): 89–97
166	HuangY M, NgM K, WenY W. A fast total variation minimization method for image restoration. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(2): 774–795 https://doi.org/10.1137/070703533
167	BressonX, ChanT F. Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems & Imaging, 2008, 2(4): 455–484 https://doi.org/10.3934/ipi.2008.2.455
168	NgM K, QiL Q, YangY F, Huang Y M. On semismooth Newton’s methods for total variation minimization. Journal of Mathematical Imaging & Vision, 2007, 27(3): 265–276 https://doi.org/10.1007/s10851-007-0650-0
169	ZhuM Q, ChanT F. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA CAM Report. 2008, 8–34
170	ZhuM Q, WrightS J, ChanT F. Duality-based algorithms for totalvariation- regularized image restoration. Computational Optimization and Applications, 2010, 47(3): 377–400 https://doi.org/10.1007/s10589-008-9225-2
171	KrishnanD, LinP, YipA M. A primal-dual active-set method for non-negativity constrained total variation deblurring problems. IEEE Transactions on Image Processing, 2007, 16(11): 2766–2777 https://doi.org/10.1109/TIP.2007.908079
172	KrishnanD, PhamQ V, YipA M. A primal-dual active-set algorithm for bilaterally constrained total variation deblurring and piecewise constant Mumford-Shah segmentation problems. Advances in Computational Mathematics, 2009, 31(1–3): 237–266 https://doi.org/10.1007/s10444-008-9101-8
173	OsherS, BurgerM, GoldfarbD, Xu J J, YinW T . An iterative regularization method for total variation-based image restoration. Multiscale Modeling & Simulation, 2005, 4(2): 460–489 https://doi.org/10.1137/040605412
174	GoldsteinT, OsherS. The split Bregman method forl1-regularized problems. SIAM Journal on Imaging Sciences, 2009, 2(2): 323–343 https://doi.org/10.1137/080725891
175	GlowinskiR, Le Tallec P. Augmented Lagrangian and Operator- Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, 1989 https://doi.org/10.1137/1.9781611970838
176	WuC C, TaiX C. Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences, 2010, 3(3): 300–339 https://doi.org/10.1137/090767558
177	DarbonJ, Sigelle M. Image restoration with discrete constrained total variation part I: fast and exact optimization. Journal of Mathematical Imaging & Vision, 2006, 26(3): 261–276 https://doi.org/10.1007/s10851-006-8803-0
178	DuanY P, TaiX C. Domain decomposition methods with graph cuts algorithms for total variation minimization. Advances in Computational Mathematics, 2012, 36(2): 175–199 https://doi.org/10.1007/s10444-011-9213-4
179	FuH Y, NgM K, NikolovaM, Barlow J L. Efficient minimization methods of mixed l2-l1 and l1-l1 norms for image restoration. SIAM Journal on Scientific Computing, 2005, 27(6): 1881–1902 https://doi.org/10.1137/040615079
180	GoldfarbD, YinW T. Second-order cone programming methods for total variation-based image restoration. SIAM Journal on Scientific Computing, 2005, 27(2): 622–645 https://doi.org/10.1137/040608982
181	OliveiraJ P, Bioucas-Dias J M, FigueiredoM A T . Adaptive total variation image deblurring: a majorization-minimization approach. Signal Processing, 2009, 89(9): 1683–1693 https://doi.org/10.1016/j.sigpro.2009.03.018
182	Bioucas-DiasJ M, Figueiredo M A T, OliveiraJ P . Total variationbased image deconvolution: a majorization-minimization approach, In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2006, 861–864
183	ChanT F, Esedoglu S. Aspects of total variation regularized l1 function approximation. SIAM Journal on Applied Mathematics, 2004, 65(5): 1817–1837 https://doi.org/10.1137/040604297
184	HeL, BurgerM, OsherS. Iterative total variation regularization with non-quadratic fidelity. Journal of Mathematical Imaging & Vision, 2006, 26(1–2): 167–184 https://doi.org/10.1007/s10851-006-8302-3
185	JonssonE, HuangS C, ChanT F. Total variation regularization in positron emission tomography. CAM Report. 1998
186	PaninV Y, ZengG L, GullbergG T . Total variation regulated EM algorithm. IEEE Transactions on Nuclear Science, 1999, 46(6): 2202–2210 https://doi.org/10.1109/23.819305
187	LeT, Chartrand R, AsakiT J . A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging & Vision, 2007, 27(3): 257–263 https://doi.org/10.1007/s10851-007-0652-y
188	RudinL, LionsP L, OsherS. Multiplicative denoising and deblurring: theory and algorithms. In: OsherS, Paragios N, eds. Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer, 2003, 103–119 https://doi.org/10.1007/0-387-21810-6_6
189	HuangY M, NgM K, WenY W. A new total variation method for multiplicative noise removal. SIAM Journal on Imaging Sciences, 2009, 2(1): 20–40 https://doi.org/10.1137/080712593
190	BoneskyT, Kazimierski K S, MaassP , SchöpferF, Schuster T. Minimization of Tikhonov functionals in Banach spaces. Abstract & Applied Analysis, 2008, 2008(1): 1563–1569 https://doi.org/10.1155/2008/192679
191	MeyerY. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series. Rhode Island: American Mathematical Society, 2002
192	BlomgrenP, ChenT F. Color TV: total variation methods for restoration of vector valued images. IEEE Transactions on Image Processing, 1970, 7(3): 304–309 https://doi.org/10.1109/83.661180
193	SetzerS, SteidlG, PopilkaB, Burgeth B. Variational methods for denoising matrix fields. In: LaidlawD , WeickertJ, eds. Visualization and Processing of Tensor Fields. Berlin: Springer Berlin Heidelberg, 2009, 341–360 https://doi.org/10.1007/978-3-540-88378-4_17
194	EsedogluS, OsherS. Decomposition of images by the anisotropic Rudin-Osher-Fatemi model. Communications on Pure and Applied Mathematics, 2004, 57(12): 1609–1626 https://doi.org/10.1002/cpa.20045
195	ShiY Y, ChangQ S. Efficient algorithm for isotropic and anisotropic total variation deblurring and denoising. Journal of Applied Mathematics, 2013 https://doi.org/10.1155/2013/797239
196	MarquinaA, OsherS. Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM Journal on Scientific Computing, 2000, 22(2): 387–405 https://doi.org/10.1137/S1064827599351751
197	ChanT F, Marquina A, MuletP . High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 2000, 22(2): 503–516 https://doi.org/10.1137/S1064827598344169
198	GilboaG, OsherS. Nonlocal operators with applications to image processing. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(3): 1005–1028 https://doi.org/10.1137/070698592
199	KindermannS, OsherS, JonesP W. Deblurring and denoising of images by nonlocal functionals. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1091–1115 https://doi.org/10.1137/050622249
200	HuY, JacobM. Higher degree total variation (HDTV) regularization for image recovery. IEEE Transactions on Image Processing, 2012, 21(5): 2559–2571 https://doi.org/10.1109/TIP.2012.2183143
201	YangJ S, YuH Y, JiangM, Wang G. High-order total variation minimization for interior SPECT. Inverse Problems, 2012, 28(1): 15001–15024 https://doi.org/10.1088/0266-5611/28/1/015001
202	LiuX W, HuangL H. A new nonlocal total variation regularization algorithm for image denoising. Mathematics and Computers in Simulation, 2014, 97: 224–233 https://doi.org/10.1016/j.matcom.2013.10.001
203	RenZ M, HeC J, ZhangQ F. Fractional order total variation regularization for image super-resolution. Signal Processing, 2013, 93(9): 2408–2421 https://doi.org/10.1016/j.sigpro.2013.02.015
204	OhS, WooH, YunS, Kang M. Non-convex hybrid total variation for image denoising. Journal of Visual Communication & Image Representation, 2013, 24(3): 332–344 https://doi.org/10.1016/j.jvcir.2013.01.010
205	DonohoD L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306 https://doi.org/10.1109/TIT.2006.871582
206	CandèE J, Wakin M B. An introduction to compressive sampling. IEEE Signal Processing Magazine, 2008, 25(2): 21–30 https://doi.org/10.1109/MSP.2007.914731
207	TsaigY, DonohoD L. Extensions of compressed sensing. Signal Processing, 2006, 86(3): 549–571 https://doi.org/10.1016/j.sigpro.2005.05.029
208	CandèsE J, Romberg J, TaoT . Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509 https://doi.org/10.1109/TIT.2005.862083
209	CandèsE J, Tao T. Near-optimal signal recovery from random projections: Universal encoding strategies?. IEEE Transactions on Information Theory, 2006, 52(12): 5406–5425 https://doi.org/10.1109/TIT.2006.885507
210	DonohoD L, EladM. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of National Academy of Sciences, 2003, 100(5): 2197–2202 https://doi.org/10.1073/pnas.0437847100
211	WrightJ, MaY. Dense error correction via l1-minimization. IEEE Transactions on Information Theory, 2010, 56(7): 3540–3560 https://doi.org/10.1109/TIT.2010.2048473
212	YangJ F, ZhangY. Alternating direction algorithms for l1-problems in compressive sensing. SIAM Journal on Scientific Computing, 2011, 33(1): 250–278. https://doi.org/10.1137/090777761
213	NatarajanB K. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 1995, 24(2): 227–234 https://doi.org/10.1137/S0097539792240406
214	MallatS G, ZhangZ. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 1993, 41(12): 3397–3415 https://doi.org/10.1109/78.258082
215	TroppJ, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 2007, 53(12): 4655–4666 https://doi.org/10.1109/TIT.2007.909108
216	BlumensathT, DaviesM E. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 629–654 https://doi.org/10.1007/s00041-008-9035-z
217	GorodnitskyI F, RaoB D. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Transactions on Signal Processing, 1997, 45(3): 600–616 https://doi.org/10.1109/78.558475
218	BaoC L, JiH, QuanY H, Shen Z W. l0 norm based dictionary learning by proximal methods with global convergence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3858–3865 https://doi.org/10.1109/cvpr.2014.493
219	FoucartS, LaiM J. Sparsest solutions of underdetermined linear systems via lq-minimization for 0<q≤1. Applied and Computational Harmonic Analysis, 2009, 26(3): 395–407 https://doi.org/10.1016/j.acha.2008.09.001
220	CaiT T, WangL, XuG. Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing, 2010, 58(3): 1300–1308 https://doi.org/10.1109/TSP.2009.2034936
221	CaiT T, WangL, XuG. New bounds for restricted isometry constants. IEEE Transactions on Information Theory, 2010, 56(9): 4388–4394 https://doi.org/10.1109/TIT.2010.2054730
222	ChenS S, DonohoD L, SaundersM A . Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 1998, 20(1): 33–61 https://doi.org/10.1137/S1064827596304010
223	EfronB, HastieT, JohnstoneI, Tibshirani R. Least angle regression. The Annals of Statistics, 2004, 32(2): 407–499 https://doi.org/10.1214/009053604000000067
224	FigueiredoM A T, Nowak R D. An EM algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 2002, 12(8): 906–916 https://doi.org/10.1109/TIP.2003.814255
225	StarckJ L, MaiK N, MurtaghF. Wavelets and curvelets for image deconvolution: a combined approach. Signal Processing, 2003, 83(10): 2279–2283 https://doi.org/10.1016/S0165-1684(03)00150-6
226	HerrholzE, Teschke G. Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems. Inverse Problems, 2010, 26(12): 125012–125035 https://doi.org/10.1088/0266-5611/26/12/125012
227	JinB, LorenzD, SchifflerS. Elastic-net regularization: error estimates and active set methods. Inverse Problems, 2009, 25(11): 1595–1610 https://doi.org/10.1088/0266-5611/25/11/115022
228	FigueiredoM A T, Nowak R D, WrightS J . Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 586–597 https://doi.org/10.1109/JSTSP.2007.910281
229	KimS J, KohK, LustigM, Boyd S, GorinevskyD . An interior-point method for large-scale l1-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 606–617 https://doi.org/10.1109/JSTSP.2007.910971
230	DonohoD L, TsaigY. Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Transactions on Information Theory, 2008, 54(11): 4789–4812 https://doi.org/10.1109/TIT.2008.929958
231	CombettesP L, WajsE R. Signal recovery by proximal forwardbackward splitting. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1168–1200 https://doi.org/10.1137/050626090
232	BeckerS, BobinJ, CandésE J . NESTA: a fast and accurate firstorder method for sparse recovery. SIAM Journal on Imaging Sciences, 2011, 4(1): 1–39 https://doi.org/10.1137/090756855
233	OsborneM R, Presnell B, TurlachB A . A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 1999, 20(3): 389–403 https://doi.org/10.1093/imanum/20.3.389
234	LiL, YaoX, StolkinR, Gong M G, HeS . An evolutionary multiobjective approach to sparse reconstruction. IEEE Transactions on Evolutionary Computation,2014, 18(6): 827–845 https://doi.org/10.1109/TEVC.2013.2287153
235	ChartrandR. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007, 14(10): 707–710 https://doi.org/10.1109/LSP.2007.898300
236	CandesE J, TaoT. Decoding by linear programming. IEEE Transactions on Information Theory, 2005, 51(12): 4203–4215 https://doi.org/10.1109/TIT.2005.858979
237	SaabR, Chartrand R, YilmazÖ . Stable sparse approximations via nonconvex optimization. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3885–3888 https://doi.org/10.1109/icassp.2008.4518502
238	TibshiraniR. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 1996, 58(1): 267–288
239	ZhangC H. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 2010, 38(2): 894–942 https://doi.org/10.1214/09-AOS729
240	FanJ Q, LiR. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 2001, 96(456): 1348–1360 https://doi.org/10.1198/016214501753382273
241	NikolovaM, NgM K, ZhangS, Ching W K. Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM Journal on Imaging Sciences, 2008, 1(1): 2–25 https://doi.org/10.1137/070692285
242	FrankL E, Friedman J H. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35(2): 109–135 https://doi.org/10.1080/00401706.1993.10485033
243	FuW J. Penalized regressions: the bridge versus the lasso. Journal of Computational and Graphical Statistics, 1998, 7(3): 397–416
244	LyuQ, LinZ C, SheY Y, Zhang C. A comparison of typical lp minimization algorithms. Neurocomputing, 2013, 119: 413–424 https://doi.org/10.1016/j.neucom.2013.03.017
245	CandesE J, WakinM B, BoydS P. Enhancing sparsity by reweighted l1 minimization. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 877–905 https://doi.org/10.1007/s00041-008-9045-x
246	RaoB D, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Transactions on Signal Processing, 1999, 47(1): 187–200 https://doi.org/10.1109/78.738251
247	SheY Y. Thresholding-based iterative selection procedures for model selection and shrinkage. Electronic Journal of Statistics, 2009, 3: 384–415 https://doi.org/10.1214/08-EJS348
248	XuZ B, ZhangH, WangY, Chang X Y, LiangY .L1/2 regularization. Science China Information Sciences, 2010, 53(6): 1159–1169 https://doi.org/10.1007/s11432-010-0090-0
249	XuZ B, GuoH L, WangY, Zhang H. Representative of L1/2 regularization among lq (0<q≤1) regularizations: an experimental study based on phase diagram. Acta Automatica Sinica, 2012, 38(7): 1225–1228 https://doi.org/10.1016/s1874-1029(11)60293-0
250	CandesE J, PlanY. Matrix completion with noise. Proceedings of the IEEE, 2009, 98(6): 925–936 https://doi.org/10.1109/JPROC.2009.2035722
251	CaiJ F, CandesE J, ShenZ. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010, 20(4): 1956–1982 https://doi.org/10.1137/080738970
252	BoydS, ParikhN, ChuE, Peleato B, EcksteinJ . Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations & Trends in Machine Learning, 2011, 3(1): 1–122 https://doi.org/10.1561/2200000016
253	QianJ J, YangJ, ZhangF L, Lin Z C. Robust low-rank regularized regression for face recognition with occlusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 21–26 https://doi.org/10.1109/cvprw.2014.9
254	LiuY J, SunD, TohK C. An implementable proximal point algorithmic framework for nuclear norm minimization. Mathematical Programming, 2012, 133(1–2): 399–436 https://doi.org/10.1007/s10107-010-0437-8
255	YangJ F, YuanX M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 2013, 82(281): 301–329 https://doi.org/10.1090/S0025-5718-2012-02598-1
256	LiT, WangW W, XuL, FengX C. Image denoising using lowrank dictionary and sparse representation. In: Proceedings of the 10th IEEE International Conference on Computational Intelligence and Security. 2014, 228–232
257	WatersA E, Sankaranarayanan A C, BaraniukR G . SpaRCS: recovering low-rank and sparse matrices from compressive measurements. In: Proceedings of the Neural Information Processing Systems Conference. 2011, 1089–1097
258	LiQ, LuZ B, LuQ B, Li H Q, LiW P . Noise reduction for hyperspectral images based on structural sparse and low-rank matrix decomposition. In: Proceedings of the IEEE International on Geoscience and Remote Sensing Symposium. 2013, 1075–1078 https://doi.org/10.1109/igarss.2013.6721350
259	ZhouT Y, TaoD C. Godec: randomized low-rank & sparse matrix decomposition in noisy case. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 33–40
260	ZhangH Y, HeW, ZhangL P, Shen H F, YuanQ Q . Hyperspectral image restoration using low-rank matrix recovery.IEEE Transactions on Geoscience & Remote Sensing, 2014, 52(8): 4729–4743 https://doi.org/10.1109/TGRS.2013.2284280
261	ZhangZ, XuY, YangJ, Li X L, ZhangD . A survey of sparse representation: algorithms and applications. IEEE Access, 2015, 3: 490–530 https://doi.org/10.1109/ACCESS.2015.2430359
262	BurgerM, FranekM, SchÖnliebC B . Regularized regression and density estimation based on optimal transport. Applied Mathematics Research eXpress, 2012, 2012(2): 209–253
263	OsherS, Solè A, VeseL . Image decomposition and restoration using total variation minimization and the H1 norm. Multiscale Modeling & Simulation, 2003, 1(3): 349–370 https://doi.org/10.1137/S1540345902416247
264	BarbaraK. Iterative regularization methods for nonlinear ill-posed problems. Algebraic Curves & Finite Fields Cryptography & Other Applications, 2008, 6
265	MiettinenK. Nonlinear Multiobjective Optimization. Springer Science & Business Media, 2012
266	MarlerR T, AroraJ S. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization, 2004, 26(6): 369–395 https://doi.org/10.1007/s00158-003-0368-6
267	GongM G, JiaoL C, YangD D, Ma W P. Research on evolutionary multi-objective optimization algorithms. Journal of Software, 2009, 20(20): 271–289 https://doi.org/10.3724/SP.J.1001.2009.00271
268	FonsecaC M, Fleming P J. Genetic algorithm for multiobjective optimization: formulation, discussion and generation. In: Proceedings of the International Conference on Genetic Algorithms. 1993, 416–423
269	SrinivasN, DebK. Multiobjective optimization using non-dominated sorting in genetic algorithms. Evolutionary Computation, 1994, 2(3): 221–248 https://doi.org/10.1162/evco.1994.2.3.221
270	HornJ, Nafpliotis N, GoldbergD E . A niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation. 1994, 1: 82–87 https://doi.org/10.1109/icec.1994.350037
271	ZitzlerE, ThieleL. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 1999, 3(4): 257–271 https://doi.org/10.1109/4235.797969
272	ZitzlerE, Laumanns M, ThieleL . SPEA2: improving the strength Pareto evolutionary algorithm. Eurogen, 2001, 3242(103): 95–100
273	KimM, Hiroyasu T, MikiM , WatanabeS. SPEA2+: improving the performance of the strength Pareto evolutionary algorithm 2. In: Proceedings of the International Conference on Parallel Problem Solving from Nature. 2004, 742–751 https://doi.org/10.1007/978-3-540-30217-9_75
274	KnowlesJ D, CorneD W. Approximating the non-dominated front using the Pareto archived evolution strategy. Evolutionary Computation, 2000, 8(2): 149–172 https://doi.org/10.1162/106365600568167
275	CorneD W, Knowles J D, OatesM J . The Pareto-envelope based selection algorithm for multi-objective optimization. In: Proceedings of the Internatioal Conference on Parallel Problem Solving from Nature. 2000, 869–878
276	CorneD W, JerramN R, KnowlesJ D, Oates M J. PESA-II: regionbased selection in evolutionary multi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference. 2001, 283–290
277	DebK, Agrawal S, PratapA , MeyarivanT. A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Lecture Notes in Computer Science, 2000, 1917: 849–858
278	ZhangQ F, LiH. MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712–731 https://doi.org/10.1109/TEVC.2007.892759
279	IshibuchiH, SakaneY, TsukamotoN, Nojima Y. Simultaneous use of different scalarizing functions in MOEA/D. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. 2010, 519–526 https://doi.org/10.1145/1830483.1830577
280	WangL P, ZhangQ F, ZhouA M, Gong M G, JiaoL C . Constrained subproblems in decomposition based multiobjective evolutionary algorithm. IEEE Transactions on Evolutionary Computation, 2016, 20(3): 475–480 https://doi.org/10.1109/TEVC.2015.2457616
281	LiK, FialhoA, KwongS, Zhang Q F. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2014, 18(1): 114–130 https://doi.org/10.1109/TEVC.2013.2239648
282	KeL J, ZhangQ F, BattitiR. Hybridization of decomposition and local search for multiobjective optimization.IEEE Transactions on Cybernetics, 2014, 44(10): 1808–1820 https://doi.org/10.1109/TCYB.2013.2295886
283	CaiX Y, WeiO. A hybrid of decomposition and domination based evolutionary algorithm for multi-objective software next release problem. In: Proceedings of the 10th IEEE International Conference on Control and Automation. 2013, 412–417 https://doi.org/10.1109/icca.2013.6565143
284	DebK, JainH. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Transactions on Evolutionary Computation , 2014, 18(4): 577–601 https://doi.org/10.1109/TEVC.2013.2281535
285	YuanY, XuH, WangB. An improved NSGA-III procedure for evolutionary many-objective optimization. In: Proceedings of ACM Annual Conference on Genetic & Evolutionary Computation. 2014, 661–668 https://doi.org/10.1145/2576768.2598342
286	SeadaH, DebK. U-NSGA-III: a unified evolutionary optimization procedure for single, multiple, and many objectives: proof-ofprinciple results. In: Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. 2015, 34–49 https://doi.org/10.1007/978-3-319-15892-1_3
287	ZhuZ X, XiaoJ, LiJ Q, Zhang Q F. Global path planning of wheeled robots using multi-objective memetic algorithms. Integrated Computer-Aided Engineering, 2015, 22(4): 387–404 https://doi.org/10.3233/ICA-150498
288	ZhuZ X, JiaS, HeS, SunY W, JiZ, ShenL L. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 2015, 298: 274–287 https://doi.org/10.1016/j.ins.2014.11.045
289	ZhuZ X, XiaoJ, HeS, JiZ, SunY W. A multi-objective memetic algorithm based on locality-sensitive hashing for one-to-many-to-one dynamic pickup-and-delivery problem. Information Sciences, 2015, 329: 73–89 https://doi.org/10.1016/j.ins.2015.09.006
290	LiH, GongM G, WangQ, Liu J, SuL Z . A multiobjective fuzzy clustering method for change detection in synthetic aperture radar images. Applied Soft Computing, 2016, 46: 767–777 https://doi.org/10.1016/j.asoc.2015.10.044
291	JinY, Sendhoff B. Pareto based approach to machine learning: an overview and case studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2008, 38(3): 397–415
292	PlumbleyM D. Recovery of sparse representations by polytope faces pursuit. In: Proceedings of the 6th International Conference on In of the first kind using singular values. SIAM Journal on Numerical Analysis, 1971, 8(3): 616–622
293	WrightS J, NowakR D, FigueiredoM A T . Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 2009, 57(7): 2479–2493 https://doi.org/10.1109/TSP.2009.2016892
294	YangY, YaoX, ZhouZ H. On the approximation ability of evolutionary optimization with application to minimum set cover. Artificial Intelligence, 2012, 180(2): 20–33
295	QianC, YuY, ZhouZ H. An analysis on recombination in multiobjective evolutionary optimization. Artificial Intelligence, 2013, 204(1): 99–119 https://doi.org/10.1016/j.artint.2013.09.002
296	GongM G, ZhangM Y, YuanY. Unsupervised band selection based on evolutionary multiobjective optimization for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(1): 544–557 https://doi.org/10.1109/TGRS.2015.2461653
297	QianC, YuY, ZhouZ H.Pareto ensemble pruning. In: Proceedings of AAAI Conference on Artificial Intelligence. 2015, 2935–2941
298	QianC, YuY, ZhouZ H. On constrained Boolean Pareto optimization. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2015, 389–395
299	QianC, YuY, ZhouZ H. Subset selection by Pareto optimization. In: Proceedings of the Neural Information Processing Systems Conference. 2015, 1765–1773
300	GongM G, LiuJ, LiH, CaiQ, SuL Z. A multiobjective sparse feature learning model for deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(12): 3263–3277 https://doi.org/10.1109/TNNLS.2015.2469673

[1]

FCS-0362-15552-MGG_suppl_1

Download

[1]	Yunyun WANG, Jiao HAN, Yating SHEN, Hui XUE. Pointwise manifold regularization for semi-supervised learning[J]. Front. Comput. Sci., 2021, 15(1): 151303-.
[2]	Jiaqing FAN, Huihui SONG, Kaihua ZHANG, Qingshan LIU, Fei YAN, Wei LIAN. Real-time manifold regularized context-aware correlation tracking[J]. Front. Comput. Sci., 2020, 14(2): 334-348.
[3]	Jie ZHANG, Xiaowei ZHAO, Meina KAN, Shiguang SHAN, Xiujuan CHAI, Xilin CHEN. Locality-constrained framework for face alignment[J]. Front. Comput. Sci., 2019, 13(4): 789-801.
[4]	Shuaiqiang WANG, Yilong YIN. Polygene-based evolutionary algorithms with frequent pattern mining[J]. Front. Comput. Sci., 2018, 12(5): 950-965.
[5]	Yong WANG, Zhi-Zhong LIU, Jianbin LI, Han-Xiong LI, Jiahai WANG. On the selection of solutions for mutation in differential evolution[J]. Front. Comput. Sci., 2018, 12(2): 297-315.
[6]	Jayanthi MANICASSAMY, Dinesh KARUNANIDHI, Sujatha POTHULA, Vengattaraman THIRUMAL, Dhavachelvan PONNURANGAM, Subramanian RAMALINGAM. GPS: a constraint-based gene position procurement in chromosome for solving large-scale multiobjective multiple knapsack problems[J]. Front. Comput. Sci., 2018, 12(1): 101-121.
[7]	Ruochen LIU,Chenlin MA,Fei HE,Wenping MA,Licheng JIAO. Reference direction based immune clone algorithm for many-objective optimization[J]. Front. Comput. Sci., 2014, 8(4): 642-655.
[8]	Yongjian ZHAO, Hong HE, Jianxun Mi. Noisy component extraction with reference[J]. Front Comput Sci, 2013, 7(1): 135-144.
[9]	Tim SCHLüTER, Stefan CONRAD. An approach for automatic sleep stage scoring and apnea-hypopnea detection[J]. Front Comput Sci, 2012, 6(2): 230-241.
[10]	Hanyang QUEK, Chunghoong WOO, Kaychen TAN, Arthur TAY. Evolving Nash-optimal poker strategies using evolutionary computation[J]. Front Comput Sci Chin, 2009, 3(1): 73-91.
[11]	Maryjane TREMAYNE, Samantha Y. CHONG, Duncan BELL. Optimisation of algorithm control parameters in cultural differential evolution applied to molecular crystallography[J]. Front Comput Sci Chin, 2009, 3(1): 101-108.
[12]	Carlos A. COELLO COELLO. Evolutionary multi-objective optimization:some current research trends and topics that remain to be explored[J]. Front Comput Sci Chin, 2009, 3(1): 18-30.

Viewed

Full text

Abstract

Cited

Shared

Discussed