Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2017, Vol. 11 Issue (3) : 362-391    https://doi.org/10.1007/s11704-016-5552-0
REVIEW ARTICLE
Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework
Maoguo GONG(), Xiangming JIANG, Hao LI
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an 710071, China
 Download: PDF(722 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Ill-posed problems are widely existed in signal processing. In this paper, we review popular regularization models such as truncated singular value decomposition regularization, iterative regularization, variational regularization. Meanwhile, we also retrospect popular optimization approaches and regularization parameter choice methods. In fact, the regularization problem is inherently a multiobjective problem. The traditional methods usually combine the fidelity term and the regularization term into a singleobjective with regularization parameters, which are difficult to tune. Therefore, we propose a multi-objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity. In this framework, the fidelity term and regularization term are optimized simultaneously to gain more insights into the ill-posed problems. A case study on signal recovery shows the effectiveness of the multi-objective framework for ill-posed problems.

Keywords ill-posed problem      regularization      multiobjective optimization      evolutionary algorithm      signal processing     
Corresponding Author(s): Maoguo GONG   
Just Accepted Date: 23 June 2016   Online First Date: 19 September 2016    Issue Date: 25 May 2017
 Cite this article:   
Maoguo GONG,Xiangming JIANG,Hao LI. Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework[J]. Front. Comput. Sci., 2017, 11(3): 362-391.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-016-5552-0
https://academic.hep.com.cn/fcs/EN/Y2017/V11/I3/362
1 HadamardJ. Sur les Problemes aux Derivees Partielles et Leur Signification Physique. Princeton University Bulletin, 1902, 13: 49–52
2 KabanikhinS I. Inverse and Ill-Posed Problems: Theory and Applications. Berlin: Water De Gruyter, 2011
https://doi.org/10.1515/9783110224016
3 ZhangB Y, XuD H, LiuT W. Stabilized algorithms for ill-posed problems in signal processing. In: Proceedings of the IEEE International Conferences on Info-tech and Info-net. 2001, 1: 375–380
https://doi.org/10.1109/ICII.2001.982776
4 ScherzerO. Handbook of Mathematical Methods in Imaging. Springer Science & Business Media, 2011
https://doi.org/10.1007/978-0-387-92920-0
5 GroetschC W. Inverse problems in the mathematical sciences. Mathematics of Computation, 1993, 63(5): 799–811
https://doi.org/10.1007/978-3-322-99202-4
6 RudinL I, OsherS, FatemiE. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 1992, 60(1): 259–268
https://doi.org/10.1016/0167-2789(92)90242-F
7 TikhonovA N. Solution of incorrectly formulated problems and the regularization method. Soviet Math, 1963, 4: 1035–1038
8 TikhonovA N, Arsenin V I. Solutions of Ill-posed Problems. Washington, DC: V. H. Winston & Sons, 1977
9 LandweberL. An iteration formula for Fredholm integral equations of the first kind. American Journal of Mathematics, 1951, 73(3): 615–624
https://doi.org/10.2307/2372313
10 HestenesM R, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409–436
https://doi.org/10.6028/jres.049.044
11 VogelC R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002, 23
https://doi.org/10.1137/1.9780898717570
12 HansenP C. The truncated SVD as a method for regularization. Bit Numerical Mathematics, 1987, 27(4): 534–553
https://doi.org/10.1007/BF01937276
13 HonerkampJ, WeeseJ. Tikhonovs regularization method for ill-posed problems. Continuum Mechanics and Thermodynamics, 1990,2(1): 17–30
https://doi.org/10.1007/BF01170953
14 ZhangX Q, BurgerM, BressonX, Osher S. Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 2010, 3(3): 253–276
https://doi.org/10.1137/090746379
15 DebK. Multi-Objective Optimization Using Evolutionary Algorithms. New York: John Wiley & Sons, 2001, 16
16 FonsecaC M, Fleming P J. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 1995, 3(1): 1–16
https://doi.org/10.1162/evco.1995.3.1.1
17 CoelloC A C, Van Veldhuizen D A, LamontG B . Evolutionary Algorithms for Solving Multi-objective Problems. New York: Kluwer Academic, 2002
https://doi.org/10.1007/978-1-4757-5184-0
18 TanK C, KhorE F, LeeT H. Multiobjective Evolutionary Algorithms and Applications. Springer Science & Business Media, 2005
19 KnowlesJ, CorneD, DebK. Multiobjective Problem Solving from Nature: from Concepts to Applications. Springer Science & Business Media, 2008
https://doi.org/10.1007/978-3-540-72964-8
20 RaquelC, YaoX. Dynamic multi-objective optimization: a survey of the state-of-the-art. In: YangS X , YaoX, eds. Evolutionary Computation for Dynamic Optimization Problems. Springer Berlin Heidelberg, 2013, 85–106
https://doi.org/10.1007/978-3-642-38416-5_4
21 LückenC V, Barán B, BrizuelaC . A survey on multi-objective evolutionary algorithms for many-objective problems. Computational Optimization and Applications, 2014, 58(3): 707–756
https://doi.org/10.1007/s10589-014-9644-1
22 HwangC L, MasudA S M.Multiple Objective Decision Making- Methods and Applications. Springer Science & Business Media, 1979, 164
https://doi.org/10.1007/978-3-642-45511-7
23 GirosiF, JonesM B, PoggioT. Regularization theory and neural networks architectures. Neural Computation, 1995, 7(2): 219–269
https://doi.org/10.1162/neco.1995.7.2.219
24 BelgeM, KilmerM E, MillerE L. Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Problems, 2002, 18(4): 1161
https://doi.org/10.1088/0266-5611/18/4/314
25 HansenP C. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. American Mathematical Monthly, 1997, 4(5): 491
26 ErikssonP, Jiménez C, BuehlerS A . Qpack, a general tool for instrument simulation and retrieval work.Journal of Quantitative Spectroscopy and Radiative Transfer, 2005, 91(1): 47–64
https://doi.org/10.1016/j.jqsrt.2004.05.050
27 GiustiE. Minimal Surfaces and Functions of Bounded Variation. Springer Science & Business Media, 1984, 80
https://doi.org/10.1007/978-1-4684-9486-0
28 CattéF, CollT.Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 1992, 29(1): 182–193
https://doi.org/10.1137/0729012
29 BjörckA. Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, 1996
https://doi.org/10.1137/1.9781611971484
30 GroetschC W. The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman Advanced Publishing Program, 1984
31 HansonR J. A numerical method for solving Fredholm integral equationstific and Statistical Computing, 1992, 13(5): 1142–1150
32 StewartG W. Rank degeneracy. SIAM Journal on Scientific and Statistical Computing, 1984, 5(2): 403–413
https://doi.org/10.1137/0905030
33 HansenP C, SekiiT, ShibahashiH . The modified truncated SVD method for regularization in general form. SIAM Journal on Sciendependent Component Analysis and Blind Source Separation. 2006, 206–213
34 Van LoanC F. Generalizing the singular value decomposition. SIAM Journal on Numerical Analysis, 1976, 13(1): 76–83
https://doi.org/10.1137/0713009
35 HansenP C.Regularization, GSVD and truncated GSVD. BIT Numerical Mathematics, 1989, 29(3): 491–504
https://doi.org/10.1007/BF02219234
36 PaigeC C. Computing the generalized singular value decomposition. SIAM Journal on Scientific and Statistical Computing, 1986, 7(4): 1126–1146
https://doi.org/10.1137/0907077
37 MorigiS, Reichel L, SgallariF . A truncated projected SVD method for linear discrete ill-posed problems. Numerical Algorithms, 2006, 43(3): 197–213
https://doi.org/10.1007/s11075-006-9053-3
38 FernandoK V, Hammarling S. A product induced singular value decomposition (ΠSVD) for two matrices and balanced realization. In: Proceedings of SIAM Conference on Linear Algebra in Signals, Systems and Control. 1988, 128–140
39 ZhaH Y. The restricted singular value decomposition of matrix triplets. SIAM Journal on Matrix Analysis and Applications, 1991, 12(1): 172–194
https://doi.org/10.1137/0612014
40 De MoorB, GolubG H. The restricted singular value decomposition: properties and applications. SIAM Journal on Matrix Analysis and Applications, 1991, 12(3): 401–425
https://doi.org/10.1137/0612029
41 De MoorB, ZhaH Y. A tree of generalizations of the ordinary singular value decomposition. Linear Algebra and Its Applications, 1991, 147: 469–500
https://doi.org/10.1016/0024-3795(91)90243-P
42 De MoorB. Generalizations of the OSVD: structure, properties and applications. In: VaccaroR J, ed.SVD & Signal Processing, II: Algorithms, Analysis & Applications. 1991, 83–98
43 NoscheseS, Reichel L. A modified TSVD method for discrete illposed problems. Numerical Linear Algebra with Applications, (in press)
44 DykesL, Noschese S, ReichelL . Rescaling the GSVD with application to ill-posed problems. Numerical Algorithms, 2015, 68(3): 531–545
https://doi.org/10.1007/s11075-014-9859-3
45 EdoL, FrancoW, MartinssonP G , RokhlinV, TygertM. Randomized algorithms for the low-rank approximation of matrices. Proceedings of the National Academy of Sciences, 2007, 104(51): 20167–20172
https://doi.org/10.1073/pnas.0709640104
46 WoolfeF, Liberty E, RokhlinV , TygertM. A fast randomized algorithm for the approximation of matrices. Applied & Computational Harmonic Analysis, 2008, 25(3): 335–366
https://doi.org/10.1016/j.acha.2007.12.002
47 SifuentesJ, Gimbutas Z, GreengardL . Randomized methods for rankdeficient linear systems. Electronic Transactions on Numerical Analysis, 2015, 44: 177–188
48 LiuY G, LeiY J, LiC G, Xu W Z, PuY F . A random algorithm for low-rank decomposition of large-scale matrices with missing entries. IEEE Transactions on Image Processing, 2015, 24(11): 4502–4511
https://doi.org/10.1109/TIP.2015.2458176
49 SekiiT. Two-dimensional inversion for solar internal rotation. Publications of the Astronomical Society of Japan, 1991, 43: 381–411
50 ScalesJ A. Uncertainties in seismic inverse calculations. In: Jacobsen B H, MosegaardK , SibaniP, eds. Inverse Methods. Berlin: Springer- Verlag, 1996, 79–97
https://doi.org/10.1007/BFb0011766
51 LawlessJ F, WangP. A simulation study of ridge and other regression estimators. Communications in Statistics-Theory and Methods, 1976, 5(4): 307–323
https://doi.org/10.1080/03610927608827353
52 DempsterA P, Schatzoff M, WermuthN . A simulation study of alternatives to ordinary least squares. Journal of the American Statistical Association, 1977, 72(357): 77–91
https://doi.org/10.1080/01621459.1977.10479910
53 HansenP C, O’Leary D P. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 1993, 14(6): 1487–1503
https://doi.org/10.1137/0914086
54 HansenP C. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Review, 1992, 34(4): 561–580
https://doi.org/10.1137/1034115
55 XuP L. Truncated SVD methods for discrete linear ill-posed problems. Geophysical Journal International, 1998, 135(2): 505–514
https://doi.org/10.1046/j.1365-246X.1998.00652.x
56 WuZ M, BianS F, XiangC B, Tong Y D. A new method for TSVD regularization truncated parameter selection. Mathematical Problems in Engineering, 2013
https://doi.org/10.1155/2013/161834
57 ChiccoD, Masseroli M. A discrete optimization approach for SVD best truncation choice based on ROC curves. In: Proceedings of the 13th IEEE International Conference on Bioinformatics and Bioengineering. 2013: 1–4
https://doi.org/10.1109/BIBE.2013.6701705
58 GolubG H, HeathM, WahbaG. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 1979, 21(2): 215–223
https://doi.org/10.1080/00401706.1979.10489751
59 JbilouK, Reichel L, SadokH . Vector extrapolation enhanced TSVD for linear discrete ill-posed problems. Numerical Algorithms, 2009, 51(2): 195–208
https://doi.org/10.1007/s11075-008-9229-0
60 BouhamidiA, JbilouK, ReichelL, Sadok H, WangZ . Vector extrapolation applied to truncated singular value decomposition and truncated iteration. Journal of Engineering Mathematics, 2015, 93(1): 99–112
https://doi.org/10.1007/s10665-013-9677-y
61 VogelC R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002
https://doi.org/10.1137/1.9780898717570
62 DoicuA, Trautmann T, SchreierF . Numerical Regularization for Atmospheric Inverse Problems. Springer Science & Business Media, 2010
https://doi.org/10.1007/978-3-642-05439-6
63 BakushinskyA B, Goncharsky A V. Iterative Methods for the Solution of Incorrect Problems. Moscow: Nauka, 1989
64 RiederA. Keine Probleme mit Inversen Problemen: Eine Einführung in ihre stabile Lösung. Berlin: Springer-Verlag, 2013
65 NemirovskiyA S, PolyakB T.Iterative methods for solving linear illposed problems under precise information. Engineering Cybernetics, 1984, 22(4): 50–56
66 BrakhageH. On ill-posed problems and the method of conjugate gradients. Inverse and Ill-posed Problems, 1987, 4: 165–175
https://doi.org/10.1016/B978-0-12-239040-1.50014-4
67 HankeM. Accelerated Landweber iterations for the solution of illposed equations. Numerische Mathematik, 1991, 60(1): 341–373
https://doi.org/10.1007/BF01385727
68 BarzilaiJ, Borwein J M. Two-point step size gradient methods. IMA Journal of Numerical Analysis, 1988, 8(1): 141–148
https://doi.org/10.1093/imanum/8.1.141
69 AxelssonO. Iterative Solution Methods. Cambridge:Cambridge University Press, 1996
70 Van der SluisA, Van der Vorst H A. The rate of convergence of conjugate gradients. Numerische Mathematik, 1986, 48(5): 543–560
https://doi.org/10.1007/BF01389450
71 ScalesJ A, Gersztenkorn A. Robust methods in inverse theory. Inverse Problems, 1988, 4(4): 1071–1091
https://doi.org/10.1088/0266-5611/4/4/010
72 BjörckÅ, Eldén L. Methods in numerical algebra for ill-posed problems. Technical Report LiTH-MAT-R-33-1979. 1979
73 TrefethenL N, BauD. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997
https://doi.org/10.1137/1.9780898719574
74 CalvettiD, LewisB, ReichelL. On the regularizing properties of the GMRES method. Numerische Mathematik, 2002, 91(4): 605–625
https://doi.org/10.1007/s002110100339
75 CalvettiD, LewisB, ReichelL. Alternating Krylov subspace image restoration methods. Journal of Computational and Applied Mathematics, 2012, 236(8): 2049–2062
https://doi.org/10.1016/j.cam.2011.09.030
76 BrianziP, FavatiP, MenchiO, Romani F. A framework for studying the regularizing properties of Krylov subspace methods.Inverse Problems, 2006, 22(3): 1007–1021
https://doi.org/10.1088/0266-5611/22/3/017
77 SonneveldP, Van Gijzen M B. IDR(s): a family of simple and fast algorithms for solving large nonsymmetric systems of linear equations. SIAM Journal on Scientific Computing, 2008, 31(2): 1035–1062
https://doi.org/10.1137/070685804
78 FongD C L, Saunders M. LSMR: an iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing, 2011, 33(5): 2950–2971
https://doi.org/10.1137/10079687X
79 ZhaoC, HuangT Z, ZhaoX L, Deng L J. Two new efficient iterative regularization methods for image restoration problems. Abstract & Applied Analysis, 2013
https://doi.org/10.1155/2013/129652
80 PerezA, Gonzalez R C. An iterative thresholding algorithm for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1987, 9(6): 742–751
https://doi.org/10.1109/TPAMI.1987.4767981
81 BeckA, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202
https://doi.org/10.1137/080716542
82 Bioucas-DiasJ M, Figueiredo M A T. Two-step algorithms for linear inverse problems with non-quadratic regularization. In: Proceedings of the IEEE International Conference on Image Processing. 2007, 105–108
https://doi.org/10.1109/icip.2007.4378902
83 Bioucas-DiasJ M, Figueiredo M A T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing. 2007, 16(12): 2992–3004
https://doi.org/10.1109/TIP.2007.909319
84 BayramI, Selesnick I W. A subband adaptive iterative shrinkage/ thresholding algorithm. IEEE Transactions on Signal Processing, 2010, 58(3): 1131–1143
https://doi.org/10.1109/TSP.2009.2036064
85 YamagishiM, YamadaI. Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize. Inverse Problems, 2011, 27(10): 105008–105022
https://doi.org/10.1088/0266-5611/27/10/105008
86 BhottoM Z A, AhmadM O, SwamyM N S. An improved fast iterative shrinkage thresholding algorithm for image deblurring. SIAM Journal on Imaging Sciences, 2015, 8(3): 1640–1657
https://doi.org/10.1137/140970537
87 ZhangY D, DongZ C, PhillipsP, Wang S H, JiG L , YangJ Q. Exponential wavelet iterative shrinkage thresholding algorithm for compressed sensing magnetic resonance imaging. Information Sciences, 2015, 322: 115–132
https://doi.org/10.1016/j.ins.2015.06.017
88 ZhangY D, WangS H, JiG L, Dong Z C. Exponential wavelet iterative shrinkage thresholding algorithm with random shift for compressed sensing magnetic resonance imaging. IEEJ Transactions on Electrical and Electronic Engineering, 2015, 10(1): 116–117
https://doi.org/10.1002/tee.22059
89 WuG M, LuoS Q. Adaptive fixed-point iterative shrinkage/ thresholding algorithm for MR imaging reconstruction using compressed sensing. Magnetic Resonance Imaging, 2014, 32(4): 372–378
https://doi.org/10.1016/j.mri.2013.12.009
90 FangE X, WangJ J, HuD F, Zhang J Y, ZouW , ZhouY. Adaptive monotone fast iterative shrinkage thresholding algorithm for fluorescence molecular tomography. IET Science Measurement Technology, 2015, 9(5): 587–595
https://doi.org/10.1049/iet-smt.2014.0030
91 ZuoW M, MengD Y, ZhangL, Feng X C, ZhangD . A generalized iterated shrinkage algorithm for non-convex sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 217–224
https://doi.org/10.1109/iccv.2013.34
92 KrishnanD, FergusR. Fast image deconvolution using hyperlaplacian priors. In: BengioY , SchuurmansD, Lafferty J D, et al., eds. Advances in Neural Information Processing Systems 22. 2009, 1033–1041
93 ChartrandR, YinW. Iteratively reweighted algorithms for compressive sensing. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3869–3872
https://doi.org/10.1109/icassp.2008.4518498
94 SheY Y. An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors. Computational Statistics & Data Analysis, 2012, 56(10): 2976–2990
https://doi.org/10.1016/j.csda.2011.11.013
95 GongP H, ZhangC S, LuZ S, Huang J Z, YeJ P . A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of International Conference on Machine Learning. 2013, 37–45
96 BrediesK, LorenzD A. Linear convergence of iterative softthresholding.Journal of Fourier Analysis and Applications, 2008, 14(5–6): 813–837
https://doi.org/10.1007/s00041-008-9041-1
97 KowalskiM. Thresholding rules and iterative shrinkage/thresholding algorithm: a convergence study. In: Proceedings of the IEEE International Conference on Image Processing. 2014, 4151–4155
https://doi.org/10.1109/icip.2014.7025843
98 ChambolleA, DossalC. On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. Journal of Optimization Theory & Applications, 2015, 166(3): 968–982
https://doi.org/10.1007/s10957-015-0746-4
99 JohnstoneP R, MoulinP. Local and global convergence of an inertial version of forward-backward splitting. Advances in Neural Information Processing Systems, 2014, 1970–1978
100 MorozovV A. On the solution of functional equations by the method of regularization. Soviet Mathematics Doklady, 1966, 7(11): 414–417
101 VainikkoG M. The discrepancy principle for a class of regularization methods. USSR Computational Mathematics and Mathematical Physics, 1982, 22(3): 1–19
https://doi.org/10.1016/0041-5553(82)90120-3
102 VainikkoG M. The critical level of discrepancy in regularization methods. USSR Computational Mathematics and Mathematical Physics, 1983, 23(6): 1–9
https://doi.org/10.1016/S0041-5553(83)80068-8
103 PlatoR. On the discrepancy principle for iterative and parametric methods to solve linear ill-posed equations. Numerische Mathematik, 1996, 75(1): 99–120
https://doi.org/10.1007/s002110050232
104 BorgesL S, Bazán F S V, CunhaM C C . Automatic stopping rule for iterative methods in discrete ill-posed problems. Computational & Applied Mathematics, 2015, 34(3): 1175–1197
https://doi.org/10.1007/s40314-014-0174-3
105 DziwokiG, Izydorczyk J. Stopping criteria analysis of the OMP algorithm for sparse channels estimation. In: Proceedings of the International Conference on Computer Networks. 2015, 250–259
https://doi.org/10.1007/978-3-319-19419-6_24
106 FavatiP, LottiG, MenchiO, Romani F.Stopping rules for iterative methods in nonnegatively constrained deconvolution. Applied Numerical Mathematics, 2014, 75: 154–166
https://doi.org/10.1016/j.apnum.2013.07.006
107 EnglH W, HankeM, NeubauerA. Regularization of Inverse Problems. Springer Science & Business Media, 1996
https://doi.org/10.1007/978-94-009-1740-8
108 AmsterP. Iterative Methods. Universitext, 2014, 53–82
https://doi.org/10.1007/978-1-4614-8893-4_3
109 WaseemM. On some iterative methods for solving system of nonlinear equations. Dissertation for the Doctoral Degree.Islamabad: COMSATS Institute of Information Technology, 2012
110 BurgerM, OsherS. A guide to the TV zoo. In: BurgerM, Mennucci A C G, OsherS , et al., eds. Level Set and PDE Based Reconstruction Methods in Imaging. Springer International Publishing, 2013, 1–70
https://doi.org/10.1007/978-3-319-01712-9_1
111 TikhonovA N. Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 1963, 4(1): 1624–1627
112 NikolovaM. Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Modeling & Simulation, 2005, 4(3): 960–991
https://doi.org/10.1137/040619582
113 BurgerM, OsherS. Convergence rates of convex variational regularization. Inverse Problems, 2004, 20(5): 1411–1421
https://doi.org/10.1088/0266-5611/20/5/005
114 HofmannB, Kaltenbacher B, PöschlC , ScherzerO. A convergence rates result for Tikhonov regularization in Banach spaces with nonsmooth operators. Inverse Problems, 2007, 23(3): 987–1010
https://doi.org/10.1088/0266-5611/23/3/009
115 ResmeritaE. Regularization of ill-posed problems in Banach spaces:convergence rates. Inverse Problems, 2005, 21(4): 1303–1314
https://doi.org/10.1088/0266-5611/21/4/007
116 ResmeritaE, Scherzer O. Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Problems, 2006, 22(3): 801–814
https://doi.org/10.1088/0266-5611/22/3/004
117 EnglH W. Discrepancy principles for Tikhonov regularization of illposed problems leading to optimal convergence rates. Journal of Optimization Theory and Applications, 1987, 52(2): 209–215
https://doi.org/10.1007/BF00941281
118 GfrererH. An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Mathematics of Computation, 1987, 49(180): 507–522
https://doi.org/10.1090/S0025-5718-1987-0906185-4
119 NattererF. Error bounds for Tikhonov regularization in Hilbert scales. Applicable Analysis, 1984, 18(1–2): 29–37
https://doi.org/10.1080/00036818408839508
120 NeubauerA. An a posteriori parameter choice for Tikhonov regularization in the presence of modeling error. Applied Numerical Mathematics, 1988, 4(6): 507–519
https://doi.org/10.1016/0168-9274(88)90013-X
121 EnglH W, Kunisch K, NeubauerA . Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Problems, 1989, 5(4): 523–540
https://doi.org/10.1088/0266-5611/5/4/007
122 ScherzerO, EnglH W, KunischK. Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM Journal on Numerical Analysis, 1993, 30(6): 1796–1838
https://doi.org/10.1137/0730091
123 VarahJ M. On the numerical solution of ill-conditioned linear systems with applications to ill-posed problems. SIAM Journal on Numerical Analysis, 1973, 10(2): 257–267
https://doi.org/10.1137/0710025
124 VinodH D, UllahA. Recent Advances in Regression Methods. Danbury: Marcel Dekker Incorporated, 1981
125 O’SullivanF. A statistical perspective on ill-posed inverse problems. Statistical Science, 1986, 1(4): 502–518
https://doi.org/10.1214/ss/1177013525
126 GrafarendE W, Schaffrin B. Ausgleichungsrechnung in linearen modellen. BI Wissenschaftsverlag Mannheim, 1993
127 RodgersC D. Inverse Methods for Atmospheric Sounding: Theory and Practice. Singapore: World Scientific, 2000
https://doi.org/10.1142/3171
128 CeccheriniS. Analytical determination of the regularization parameter in the retrieval of atmospheric vertical profiles. Optics Letters, 2005, 30(19): 2554–2556
https://doi.org/10.1364/OL.30.002554
129 MallowsC L. Some comments on Cp. Technometrics, 1973, 15(4): 661–675
130 RiceJ. Choice of smoothing parameter in deconvolution problems. Contemporary Mathematics, 1986, 59: 137–151
https://doi.org/10.1090/conm/059/10
131 HankeM, RausT. A general heuristic for choosing the regularization parameter in ill-posed problems. SIAM Journal on Scientific Computing, 1996, 17(4): 956–972
https://doi.org/10.1137/0917062
132 WuL M. A parameter choice method for Tikhonov regularization. Electronic Transactions on Numerical Analysis, 2003, 16: 107–128
133 GaoW, YuK P. A new method for determining the Tikhonov regularization parameter of load identification. In: Proceedings of the International Symposium on Precision Engineering Measurement and Instrumentation. 2015
134 ItoK, JinB, TakeuchiT. Multi-parameter Tikhonov regularizationan augmented approach. Chinese Annals of Mathematics, Series B, 2014, 35(03): 383–398
https://doi.org/10.1007/s11401-014-0835-y
135 JinB, LorenzD A. Heuristic parameter-choice rules for convex variational regularization based on error estimates. SIAM Journal on Numerical Analysis, 2010, 48(3): 1208–1229
https://doi.org/10.1137/100784369
136 PazosF, BhayaA. Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov optimizing control. Journal of Computational and Applied Mathematics, 2015, 279: 123–132
https://doi.org/10.1016/j.cam.2014.10.022
137 HämarikU, PalmR, RausT. A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level. Journal of Computational and Applied Mathematics, 2012, 236(8): 2146–2157
https://doi.org/10.1016/j.cam.2011.09.037
138 ReichelL, Rodriguez G. Old and new parameter choice rules for discrete ill-posed problems. Numerical Algorithms, 2013, 63(1): 65–87
https://doi.org/10.1007/s11075-012-9612-8
139 KryanevA V. An iterative method for solving incorrectly posed problems. USSR Computational Mathematics and Mathematical Physics, 1974, 14(1): 24–35
https://doi.org/10.1016/0041-5553(74)90133-5
140 KingJ T, Chillingworth D. Approximation of generalized inverses by iterated regularization. Numerical Functional Analysis & Optimization, 1979, 1(5): 499–513
https://doi.org/10.1080/01630567908816031
141 FakeevA G. A class of iterative processes for solving degenerate systems of linear algebraic equations. USSR Computational Mathematics and Mathematical Physics, 1981, 21(3): 15–22
https://doi.org/10.1016/0041-5553(81)90060-4
142 BrillM, SchockE. Iterative solution of ill-posed problems: a survey. In: Proceedings of the 4th International Mathematical Geophysics Seminar. 1987
143 HankeM, Groetsch C W. Nonstationary iterated Tikhonov regularization. Journal of Optimization Theory and Applications, 1998, 98(1): 37–53
https://doi.org/10.1023/A:1022680629327
144 LampeJ, Reichel L, VossH . Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra and Its Applications, 2012, 436(8): 2845–2865
https://doi.org/10.1016/j.laa.2011.07.019
145 ReichelL, YuX B. Tikhonov regularization via flexible Arnoldi reduction. BIT Numerical Mathematics, 2015, 55(4): 1145–1168
https://doi.org/10.1007/s10543-014-0542-9
146 HuangG, Reichel L, YinF . Projected nonstationary iterated Tikhonov regularization. BIT Numerical Mathematics, 2016, 56(2): 467–487
https://doi.org/10.1007/s10543-015-0568-7
147 AmbrosioL, FuscoN, PallaraD. Functions of Bounded Variation and Free Discontinuity Problems. Oxford: Oxford University Press, 2000
148 AcarR, VogelC R. Analysis of bounded variation penalty methods for ill-posed problems. Inverse Problems, 1997, 10(6): 1217–1229
https://doi.org/10.1088/0266-5611/10/6/003
149 HuntB R. The application of constrained least squares estimation to image restoration by digital computer. IEEE Transactions on Computers, 1973, 100(9): 805–812
https://doi.org/10.1109/TC.1973.5009169
150 DemomentG. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech and Signal Processing, 1989, 37(12): 2024–2036
https://doi.org/10.1109/29.45551
151 KatsaggelosA K. Iterative image restoration algorithms. Optical Engineering, 1989, 28(7): 735–748
https://doi.org/10.1117/12.7977030
152 KatsaggelosA K, Biemond J, SchaferR W , MersereauR M. A regularized iterative image restoration algorithm. IEEE Transactions on Signal Processing, 1991, 39(4): 914–929
https://doi.org/10.1109/78.80914
153 BabacanS D, MolinaR, KatsaggelosA K . Parameter estimation in TV image restoration using variational distribution approximation. IEEE Transactions on Image Processing, 2008, 17(3): 326–339
https://doi.org/10.1109/TIP.2007.916051
154 WenY W, ChanR H. Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Transactions on Image Processing, 2012, 21(4): 1770–1781
https://doi.org/10.1109/TIP.2011.2181401
155 ChenA, HuoB M, WenC W. Adaptive regularization for color image restoration using discrepancy principle. In: Proceedings of the IEEE International Conference on Signal processing, Comminications and Computing. 2013, 1–6
https://doi.org/10.1109/icspcc.2013.6663988
156 LinY, Wohlberg B, GuoH . UPRE method for total variation parameter selection. Signal Processing, 2010, 90(8): 2546–2551
https://doi.org/10.1016/j.sigpro.2010.02.025
157 SteinC M. Estimation of the mean of a multivariate normal distribution. Annals of Statistics, 1981, 9(6): 1135–1151
https://doi.org/10.1214/aos/1176345632
158 RamaniS, BluT, UnserM. Monte-Carlo SURE: a black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on Image Processing, 2008, 17(9): 1540–1554
https://doi.org/10.1109/TIP.2008.2001404
159 PalssonF, Sveinsson J R, UlfarssonM O , BenediktssonJ A. SAR image denoising using total variation based regularization with surebased optimization of regularization parameter. In: Proceedings of the IEEE International Conference on Geoscience and Remote Sensing Symposium. 2012, 2160–2163
160 LiaoH Y, LiF, NgM K. Selection of regularization parameter in total variation image restoration. Journal of the Optical Society of America A, 2009, 26(11): 2311–2320
https://doi.org/10.1364/JOSAA.26.002311
161 BertalmíoM, Caselles V, RougéB , SoléA. TV based image restoration with local constraints. Journal of Scientific Computing, 2003, 19(1–3): 95–122
https://doi.org/10.1023/A:1025391506181
162 AlmansaA, Ballester C, CasellesV , HaroG. A TV based restoration model with local constraints. Journal of Scientific Computing, 2008, 34(3): 209–236
https://doi.org/10.1007/s10915-007-9160-x
163 VogelC R, OmanM E. Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 1997, 17(1): 227–238
https://doi.org/10.1137/0917016
164 ChanT F, GolubG H, MuletP. A nonlinear primal-dual method for total variation-based image restoration. Lecture Notes in Control & Information Sciences, 1995, 20(6): 1964–1977
165 ChambolleA. An algorithm for total variation minimization and applications. Journal ofMathematical Imaging & Vision, 2004, 20(1–2): 89–97
166 HuangY M, NgM K, WenY W. A fast total variation minimization method for image restoration. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(2): 774–795
https://doi.org/10.1137/070703533
167 BressonX, ChanT F. Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems & Imaging, 2008, 2(4): 455–484
https://doi.org/10.3934/ipi.2008.2.455
168 NgM K, QiL Q, YangY F, Huang Y M. On semismooth Newton’s methods for total variation minimization. Journal of Mathematical Imaging & Vision, 2007, 27(3): 265–276
https://doi.org/10.1007/s10851-007-0650-0
169 ZhuM Q, ChanT F. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA CAM Report. 2008, 8–34
170 ZhuM Q, WrightS J, ChanT F. Duality-based algorithms for totalvariation- regularized image restoration. Computational Optimization and Applications, 2010, 47(3): 377–400
https://doi.org/10.1007/s10589-008-9225-2
171 KrishnanD, LinP, YipA M. A primal-dual active-set method for non-negativity constrained total variation deblurring problems. IEEE Transactions on Image Processing, 2007, 16(11): 2766–2777
https://doi.org/10.1109/TIP.2007.908079
172 KrishnanD, PhamQ V, YipA M. A primal-dual active-set algorithm for bilaterally constrained total variation deblurring and piecewise constant Mumford-Shah segmentation problems. Advances in Computational Mathematics, 2009, 31(1–3): 237–266
https://doi.org/10.1007/s10444-008-9101-8
173 OsherS, BurgerM, GoldfarbD, Xu J J, YinW T . An iterative regularization method for total variation-based image restoration. Multiscale Modeling & Simulation, 2005, 4(2): 460–489
https://doi.org/10.1137/040605412
174 GoldsteinT, OsherS. The split Bregman method forl1-regularized problems. SIAM Journal on Imaging Sciences, 2009, 2(2): 323–343
https://doi.org/10.1137/080725891
175 GlowinskiR, Le Tallec P. Augmented Lagrangian and Operator- Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, 1989
https://doi.org/10.1137/1.9781611970838
176 WuC C, TaiX C. Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences, 2010, 3(3): 300–339
https://doi.org/10.1137/090767558
177 DarbonJ, Sigelle M. Image restoration with discrete constrained total variation part I: fast and exact optimization. Journal of Mathematical Imaging & Vision, 2006, 26(3): 261–276
https://doi.org/10.1007/s10851-006-8803-0
178 DuanY P, TaiX C. Domain decomposition methods with graph cuts algorithms for total variation minimization. Advances in Computational Mathematics, 2012, 36(2): 175–199
https://doi.org/10.1007/s10444-011-9213-4
179 FuH Y, NgM K, NikolovaM, Barlow J L. Efficient minimization methods of mixed l2-l1 and l1-l1 norms for image restoration. SIAM Journal on Scientific Computing, 2005, 27(6): 1881–1902
https://doi.org/10.1137/040615079
180 GoldfarbD, YinW T. Second-order cone programming methods for total variation-based image restoration. SIAM Journal on Scientific Computing, 2005, 27(2): 622–645
https://doi.org/10.1137/040608982
181 OliveiraJ P, Bioucas-Dias J M, FigueiredoM A T . Adaptive total variation image deblurring: a majorization-minimization approach. Signal Processing, 2009, 89(9): 1683–1693
https://doi.org/10.1016/j.sigpro.2009.03.018
182 Bioucas-DiasJ M, Figueiredo M A T, OliveiraJ P . Total variationbased image deconvolution: a majorization-minimization approach, In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2006, 861–864
183 ChanT F, Esedoglu S. Aspects of total variation regularized l1 function approximation. SIAM Journal on Applied Mathematics, 2004, 65(5): 1817–1837
https://doi.org/10.1137/040604297
184 HeL, BurgerM, OsherS. Iterative total variation regularization with non-quadratic fidelity. Journal of Mathematical Imaging & Vision, 2006, 26(1–2): 167–184
https://doi.org/10.1007/s10851-006-8302-3
185 JonssonE, HuangS C, ChanT F. Total variation regularization in positron emission tomography. CAM Report. 1998
186 PaninV Y, ZengG L, GullbergG T . Total variation regulated EM algorithm. IEEE Transactions on Nuclear Science, 1999, 46(6): 2202–2210
https://doi.org/10.1109/23.819305
187 LeT, Chartrand R, AsakiT J . A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging & Vision, 2007, 27(3): 257–263
https://doi.org/10.1007/s10851-007-0652-y
188 RudinL, LionsP L, OsherS. Multiplicative denoising and deblurring: theory and algorithms. In: OsherS, Paragios N, eds. Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer, 2003, 103–119
https://doi.org/10.1007/0-387-21810-6_6
189 HuangY M, NgM K, WenY W. A new total variation method for multiplicative noise removal. SIAM Journal on Imaging Sciences, 2009, 2(1): 20–40
https://doi.org/10.1137/080712593
190 BoneskyT, Kazimierski K S, MaassP , SchöpferF, Schuster T. Minimization of Tikhonov functionals in Banach spaces. Abstract & Applied Analysis, 2008, 2008(1): 1563–1569
https://doi.org/10.1155/2008/192679
191 MeyerY. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series. Rhode Island: American Mathematical Society, 2002
192 BlomgrenP, ChenT F. Color TV: total variation methods for restoration of vector valued images. IEEE Transactions on Image Processing, 1970, 7(3): 304–309
https://doi.org/10.1109/83.661180
193 SetzerS, SteidlG, PopilkaB, Burgeth B. Variational methods for denoising matrix fields. In: LaidlawD , WeickertJ, eds. Visualization and Processing of Tensor Fields. Berlin: Springer Berlin Heidelberg, 2009, 341–360
https://doi.org/10.1007/978-3-540-88378-4_17
194 EsedogluS, OsherS. Decomposition of images by the anisotropic Rudin-Osher-Fatemi model. Communications on Pure and Applied Mathematics, 2004, 57(12): 1609–1626
https://doi.org/10.1002/cpa.20045
195 ShiY Y, ChangQ S. Efficient algorithm for isotropic and anisotropic total variation deblurring and denoising. Journal of Applied Mathematics, 2013
https://doi.org/10.1155/2013/797239
196 MarquinaA, OsherS. Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM Journal on Scientific Computing, 2000, 22(2): 387–405
https://doi.org/10.1137/S1064827599351751
197 ChanT F, Marquina A, MuletP . High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 2000, 22(2): 503–516
https://doi.org/10.1137/S1064827598344169
198 GilboaG, OsherS. Nonlocal operators with applications to image processing. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(3): 1005–1028
https://doi.org/10.1137/070698592
199 KindermannS, OsherS, JonesP W. Deblurring and denoising of images by nonlocal functionals. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1091–1115
https://doi.org/10.1137/050622249
200 HuY, JacobM. Higher degree total variation (HDTV) regularization for image recovery. IEEE Transactions on Image Processing, 2012, 21(5): 2559–2571
https://doi.org/10.1109/TIP.2012.2183143
201 YangJ S, YuH Y, JiangM, Wang G. High-order total variation minimization for interior SPECT. Inverse Problems, 2012, 28(1): 15001–15024
https://doi.org/10.1088/0266-5611/28/1/015001
202 LiuX W, HuangL H. A new nonlocal total variation regularization algorithm for image denoising. Mathematics and Computers in Simulation, 2014, 97: 224–233
https://doi.org/10.1016/j.matcom.2013.10.001
203 RenZ M, HeC J, ZhangQ F. Fractional order total variation regularization for image super-resolution. Signal Processing, 2013, 93(9): 2408–2421
https://doi.org/10.1016/j.sigpro.2013.02.015
204 OhS, WooH, YunS, Kang M. Non-convex hybrid total variation for image denoising. Journal of Visual Communication & Image Representation, 2013, 24(3): 332–344
https://doi.org/10.1016/j.jvcir.2013.01.010
205 DonohoD L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306
https://doi.org/10.1109/TIT.2006.871582
206 CandèE J, Wakin M B. An introduction to compressive sampling. IEEE Signal Processing Magazine, 2008, 25(2): 21–30
https://doi.org/10.1109/MSP.2007.914731
207 TsaigY, DonohoD L. Extensions of compressed sensing. Signal Processing, 2006, 86(3): 549–571
https://doi.org/10.1016/j.sigpro.2005.05.029
208 CandèsE J, Romberg J, TaoT . Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509
https://doi.org/10.1109/TIT.2005.862083
209 CandèsE J, Tao T. Near-optimal signal recovery from random projections: Universal encoding strategies?. IEEE Transactions on Information Theory, 2006, 52(12): 5406–5425
https://doi.org/10.1109/TIT.2006.885507
210 DonohoD L, EladM. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of National Academy of Sciences, 2003, 100(5): 2197–2202
https://doi.org/10.1073/pnas.0437847100
211 WrightJ, MaY. Dense error correction via l1-minimization. IEEE Transactions on Information Theory, 2010, 56(7): 3540–3560
https://doi.org/10.1109/TIT.2010.2048473
212 YangJ F, ZhangY. Alternating direction algorithms for l1-problems in compressive sensing. SIAM Journal on Scientific Computing, 2011, 33(1): 250–278.
https://doi.org/10.1137/090777761
213 NatarajanB K. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 1995, 24(2): 227–234
https://doi.org/10.1137/S0097539792240406
214 MallatS G, ZhangZ. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 1993, 41(12): 3397–3415
https://doi.org/10.1109/78.258082
215 TroppJ, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 2007, 53(12): 4655–4666
https://doi.org/10.1109/TIT.2007.909108
216 BlumensathT, DaviesM E. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 629–654
https://doi.org/10.1007/s00041-008-9035-z
217 GorodnitskyI F, RaoB D. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Transactions on Signal Processing, 1997, 45(3): 600–616
https://doi.org/10.1109/78.558475
218 BaoC L, JiH, QuanY H, Shen Z W. l0 norm based dictionary learning by proximal methods with global convergence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3858–3865
https://doi.org/10.1109/cvpr.2014.493
219 FoucartS, LaiM J. Sparsest solutions of underdetermined linear systems via lq-minimization for 0<q≤1. Applied and Computational Harmonic Analysis, 2009, 26(3): 395–407
https://doi.org/10.1016/j.acha.2008.09.001
220 CaiT T, WangL, XuG. Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing, 2010, 58(3): 1300–1308
https://doi.org/10.1109/TSP.2009.2034936
221 CaiT T, WangL, XuG. New bounds for restricted isometry constants. IEEE Transactions on Information Theory, 2010, 56(9): 4388–4394
https://doi.org/10.1109/TIT.2010.2054730
222 ChenS S, DonohoD L, SaundersM A . Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 1998, 20(1): 33–61
https://doi.org/10.1137/S1064827596304010
223 EfronB, HastieT, JohnstoneI, Tibshirani R. Least angle regression. The Annals of Statistics, 2004, 32(2): 407–499
https://doi.org/10.1214/009053604000000067
224 FigueiredoM A T, Nowak R D. An EM algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 2002, 12(8): 906–916
https://doi.org/10.1109/TIP.2003.814255
225 StarckJ L, MaiK N, MurtaghF. Wavelets and curvelets for image deconvolution: a combined approach. Signal Processing, 2003, 83(10): 2279–2283
https://doi.org/10.1016/S0165-1684(03)00150-6
226 HerrholzE, Teschke G. Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems. Inverse Problems, 2010, 26(12): 125012–125035
https://doi.org/10.1088/0266-5611/26/12/125012
227 JinB, LorenzD, SchifflerS. Elastic-net regularization: error estimates and active set methods. Inverse Problems, 2009, 25(11): 1595–1610
https://doi.org/10.1088/0266-5611/25/11/115022
228 FigueiredoM A T, Nowak R D, WrightS J . Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 586–597
https://doi.org/10.1109/JSTSP.2007.910281
229 KimS J, KohK, LustigM, Boyd S, GorinevskyD . An interior-point method for large-scale l1-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 606–617
https://doi.org/10.1109/JSTSP.2007.910971
230 DonohoD L, TsaigY. Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Transactions on Information Theory, 2008, 54(11): 4789–4812
https://doi.org/10.1109/TIT.2008.929958
231 CombettesP L, WajsE R. Signal recovery by proximal forwardbackward splitting. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1168–1200
https://doi.org/10.1137/050626090
232 BeckerS, BobinJ, CandésE J . NESTA: a fast and accurate firstorder method for sparse recovery. SIAM Journal on Imaging Sciences, 2011, 4(1): 1–39
https://doi.org/10.1137/090756855
233 OsborneM R, Presnell B, TurlachB A . A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 1999, 20(3): 389–403
https://doi.org/10.1093/imanum/20.3.389
234 LiL, YaoX, StolkinR, Gong M G, HeS . An evolutionary multiobjective approach to sparse reconstruction. IEEE Transactions on Evolutionary Computation,2014, 18(6): 827–845
https://doi.org/10.1109/TEVC.2013.2287153
235 ChartrandR. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007, 14(10): 707–710
https://doi.org/10.1109/LSP.2007.898300
236 CandesE J, TaoT. Decoding by linear programming. IEEE Transactions on Information Theory, 2005, 51(12): 4203–4215
https://doi.org/10.1109/TIT.2005.858979
237 SaabR, Chartrand R, YilmazÖ . Stable sparse approximations via nonconvex optimization. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3885–3888
https://doi.org/10.1109/icassp.2008.4518502
238 TibshiraniR. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 1996, 58(1): 267–288
239 ZhangC H. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 2010, 38(2): 894–942
https://doi.org/10.1214/09-AOS729
240 FanJ Q, LiR. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 2001, 96(456): 1348–1360
https://doi.org/10.1198/016214501753382273
241 NikolovaM, NgM K, ZhangS, Ching W K. Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM Journal on Imaging Sciences, 2008, 1(1): 2–25
https://doi.org/10.1137/070692285
242 FrankL E, Friedman J H. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35(2): 109–135
https://doi.org/10.1080/00401706.1993.10485033
243 FuW J. Penalized regressions: the bridge versus the lasso. Journal of Computational and Graphical Statistics, 1998, 7(3): 397–416
244 LyuQ, LinZ C, SheY Y, Zhang C. A comparison of typical lp minimization algorithms. Neurocomputing, 2013, 119: 413–424
https://doi.org/10.1016/j.neucom.2013.03.017
245 CandesE J, WakinM B, BoydS P. Enhancing sparsity by reweighted l1 minimization. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 877–905
https://doi.org/10.1007/s00041-008-9045-x
246 RaoB D, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Transactions on Signal Processing, 1999, 47(1): 187–200
https://doi.org/10.1109/78.738251
247 SheY Y. Thresholding-based iterative selection procedures for model selection and shrinkage. Electronic Journal of Statistics, 2009, 3: 384–415
https://doi.org/10.1214/08-EJS348
248 XuZ B, ZhangH, WangY, Chang X Y, LiangY .L1/2 regularization. Science China Information Sciences, 2010, 53(6): 1159–1169
https://doi.org/10.1007/s11432-010-0090-0
249 XuZ B, GuoH L, WangY, Zhang H. Representative of L1/2 regularization among lq (0<q≤1) regularizations: an experimental study based on phase diagram. Acta Automatica Sinica, 2012, 38(7): 1225–1228
https://doi.org/10.1016/s1874-1029(11)60293-0
250 CandesE J, PlanY. Matrix completion with noise. Proceedings of the IEEE, 2009, 98(6): 925–936
https://doi.org/10.1109/JPROC.2009.2035722
251 CaiJ F, CandesE J, ShenZ. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010, 20(4): 1956–1982
https://doi.org/10.1137/080738970
252 BoydS, ParikhN, ChuE, Peleato B, EcksteinJ . Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations & Trends in Machine Learning, 2011, 3(1): 1–122
https://doi.org/10.1561/2200000016
253 QianJ J, YangJ, ZhangF L, Lin Z C. Robust low-rank regularized regression for face recognition with occlusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 21–26
https://doi.org/10.1109/cvprw.2014.9
254 LiuY J, SunD, TohK C. An implementable proximal point algorithmic framework for nuclear norm minimization. Mathematical Programming, 2012, 133(1–2): 399–436
https://doi.org/10.1007/s10107-010-0437-8
255 YangJ F, YuanX M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 2013, 82(281): 301–329
https://doi.org/10.1090/S0025-5718-2012-02598-1
256 LiT, WangW W, XuL, FengX C. Image denoising using lowrank dictionary and sparse representation. In: Proceedings of the 10th IEEE International Conference on Computational Intelligence and Security. 2014, 228–232
257 WatersA E, Sankaranarayanan A C, BaraniukR G . SpaRCS: recovering low-rank and sparse matrices from compressive measurements. In: Proceedings of the Neural Information Processing Systems Conference. 2011, 1089–1097
258 LiQ, LuZ B, LuQ B, Li H Q, LiW P . Noise reduction for hyperspectral images based on structural sparse and low-rank matrix decomposition. In: Proceedings of the IEEE International on Geoscience and Remote Sensing Symposium. 2013, 1075–1078
https://doi.org/10.1109/igarss.2013.6721350
259 ZhouT Y, TaoD C. Godec: randomized low-rank & sparse matrix decomposition in noisy case. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 33–40
260 ZhangH Y, HeW, ZhangL P, Shen H F, YuanQ Q . Hyperspectral image restoration using low-rank matrix recovery.IEEE Transactions on Geoscience & Remote Sensing, 2014, 52(8): 4729–4743
https://doi.org/10.1109/TGRS.2013.2284280
261 ZhangZ, XuY, YangJ, Li X L, ZhangD . A survey of sparse representation: algorithms and applications. IEEE Access, 2015, 3: 490–530
https://doi.org/10.1109/ACCESS.2015.2430359
262 BurgerM, FranekM, SchÖnliebC B . Regularized regression and density estimation based on optimal transport. Applied Mathematics Research eXpress, 2012, 2012(2): 209–253
263 OsherS, Solè A, VeseL . Image decomposition and restoration using total variation minimization and the H1 norm. Multiscale Modeling & Simulation, 2003, 1(3): 349–370
https://doi.org/10.1137/S1540345902416247
264 BarbaraK. Iterative regularization methods for nonlinear ill-posed problems. Algebraic Curves & Finite Fields Cryptography & Other Applications, 2008, 6
265 MiettinenK. Nonlinear Multiobjective Optimization. Springer Science & Business Media, 2012
266 MarlerR T, AroraJ S. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization, 2004, 26(6): 369–395
https://doi.org/10.1007/s00158-003-0368-6
267 GongM G, JiaoL C, YangD D, Ma W P. Research on evolutionary multi-objective optimization algorithms. Journal of Software, 2009, 20(20): 271–289
https://doi.org/10.3724/SP.J.1001.2009.00271
268 FonsecaC M, Fleming P J. Genetic algorithm for multiobjective optimization: formulation, discussion and generation. In: Proceedings of the International Conference on Genetic Algorithms. 1993, 416–423
269 SrinivasN, DebK. Multiobjective optimization using non-dominated sorting in genetic algorithms. Evolutionary Computation, 1994, 2(3): 221–248
https://doi.org/10.1162/evco.1994.2.3.221
270 HornJ, Nafpliotis N, GoldbergD E . A niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation. 1994, 1: 82–87
https://doi.org/10.1109/icec.1994.350037
271 ZitzlerE, ThieleL. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 1999, 3(4): 257–271
https://doi.org/10.1109/4235.797969
272 ZitzlerE, Laumanns M, ThieleL . SPEA2: improving the strength Pareto evolutionary algorithm. Eurogen, 2001, 3242(103): 95–100
273 KimM, Hiroyasu T, MikiM , WatanabeS. SPEA2+: improving the performance of the strength Pareto evolutionary algorithm 2. In: Proceedings of the International Conference on Parallel Problem Solving from Nature. 2004, 742–751
https://doi.org/10.1007/978-3-540-30217-9_75
274 KnowlesJ D, CorneD W. Approximating the non-dominated front using the Pareto archived evolution strategy. Evolutionary Computation, 2000, 8(2): 149–172
https://doi.org/10.1162/106365600568167
275 CorneD W, Knowles J D, OatesM J . The Pareto-envelope based selection algorithm for multi-objective optimization. In: Proceedings of the Internatioal Conference on Parallel Problem Solving from Nature. 2000, 869–878
276 CorneD W, JerramN R, KnowlesJ D, Oates M J. PESA-II: regionbased selection in evolutionary multi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference. 2001, 283–290
277 DebK, Agrawal S, PratapA , MeyarivanT. A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Lecture Notes in Computer Science, 2000, 1917: 849–858
278 ZhangQ F, LiH. MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712–731
https://doi.org/10.1109/TEVC.2007.892759
279 IshibuchiH, SakaneY, TsukamotoN, Nojima Y. Simultaneous use of different scalarizing functions in MOEA/D. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. 2010, 519–526
https://doi.org/10.1145/1830483.1830577
280 WangL P, ZhangQ F, ZhouA M, Gong M G, JiaoL C . Constrained subproblems in decomposition based multiobjective evolutionary algorithm. IEEE Transactions on Evolutionary Computation, 2016, 20(3): 475–480
https://doi.org/10.1109/TEVC.2015.2457616
281 LiK, FialhoA, KwongS, Zhang Q F. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2014, 18(1): 114–130
https://doi.org/10.1109/TEVC.2013.2239648
282 KeL J, ZhangQ F, BattitiR. Hybridization of decomposition and local search for multiobjective optimization.IEEE Transactions on Cybernetics, 2014, 44(10): 1808–1820
https://doi.org/10.1109/TCYB.2013.2295886
283 CaiX Y, WeiO. A hybrid of decomposition and domination based evolutionary algorithm for multi-objective software next release problem. In: Proceedings of the 10th IEEE International Conference on Control and Automation. 2013, 412–417
https://doi.org/10.1109/icca.2013.6565143
284 DebK, JainH. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Transactions on Evolutionary Computation , 2014, 18(4): 577–601
https://doi.org/10.1109/TEVC.2013.2281535
285 YuanY, XuH, WangB. An improved NSGA-III procedure for evolutionary many-objective optimization. In: Proceedings of ACM Annual Conference on Genetic & Evolutionary Computation. 2014, 661–668
https://doi.org/10.1145/2576768.2598342
286 SeadaH, DebK. U-NSGA-III: a unified evolutionary optimization procedure for single, multiple, and many objectives: proof-ofprinciple results. In: Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. 2015, 34–49
https://doi.org/10.1007/978-3-319-15892-1_3
287 ZhuZ X, XiaoJ, LiJ Q, Zhang Q F. Global path planning of wheeled robots using multi-objective memetic algorithms. Integrated Computer-Aided Engineering, 2015, 22(4): 387–404
https://doi.org/10.3233/ICA-150498
288 ZhuZ X, JiaS, HeS, SunY W, JiZ, ShenL L. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 2015, 298: 274–287
https://doi.org/10.1016/j.ins.2014.11.045
289 ZhuZ X, XiaoJ, HeS, JiZ, SunY W. A multi-objective memetic algorithm based on locality-sensitive hashing for one-to-many-to-one dynamic pickup-and-delivery problem. Information Sciences, 2015, 329: 73–89
https://doi.org/10.1016/j.ins.2015.09.006
290 LiH, GongM G, WangQ, Liu J, SuL Z . A multiobjective fuzzy clustering method for change detection in synthetic aperture radar images. Applied Soft Computing, 2016, 46: 767–777
https://doi.org/10.1016/j.asoc.2015.10.044
291 JinY, Sendhoff B. Pareto based approach to machine learning: an overview and case studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2008, 38(3): 397–415
292 PlumbleyM D. Recovery of sparse representations by polytope faces pursuit. In: Proceedings of the 6th International Conference on In of the first kind using singular values. SIAM Journal on Numerical Analysis, 1971, 8(3): 616–622
293 WrightS J, NowakR D, FigueiredoM A T . Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 2009, 57(7): 2479–2493
https://doi.org/10.1109/TSP.2009.2016892
294 YangY, YaoX, ZhouZ H. On the approximation ability of evolutionary optimization with application to minimum set cover. Artificial Intelligence, 2012, 180(2): 20–33
295 QianC, YuY, ZhouZ H. An analysis on recombination in multiobjective evolutionary optimization. Artificial Intelligence, 2013, 204(1): 99–119
https://doi.org/10.1016/j.artint.2013.09.002
296 GongM G, ZhangM Y, YuanY. Unsupervised band selection based on evolutionary multiobjective optimization for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(1): 544–557
https://doi.org/10.1109/TGRS.2015.2461653
297 QianC, YuY, ZhouZ H.Pareto ensemble pruning. In: Proceedings of AAAI Conference on Artificial Intelligence. 2015, 2935–2941
298 QianC, YuY, ZhouZ H. On constrained Boolean Pareto optimization. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2015, 389–395
299 QianC, YuY, ZhouZ H. Subset selection by Pareto optimization. In: Proceedings of the Neural Information Processing Systems Conference. 2015, 1765–1773
300 GongM G, LiuJ, LiH, CaiQ, SuL Z. A multiobjective sparse feature learning model for deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(12): 3263–3277
https://doi.org/10.1109/TNNLS.2015.2469673
[1] FCS-0362-15552-MGG_suppl_1 Download
[1] Yunyun WANG, Jiao HAN, Yating SHEN, Hui XUE. Pointwise manifold regularization for semi-supervised learning[J]. Front. Comput. Sci., 2021, 15(1): 151303-.
[2] Jiaqing FAN, Huihui SONG, Kaihua ZHANG, Qingshan LIU, Fei YAN, Wei LIAN. Real-time manifold regularized context-aware correlation tracking[J]. Front. Comput. Sci., 2020, 14(2): 334-348.
[3] Jie ZHANG, Xiaowei ZHAO, Meina KAN, Shiguang SHAN, Xiujuan CHAI, Xilin CHEN. Locality-constrained framework for face alignment[J]. Front. Comput. Sci., 2019, 13(4): 789-801.
[4] Shuaiqiang WANG, Yilong YIN. Polygene-based evolutionary algorithms with frequent pattern mining[J]. Front. Comput. Sci., 2018, 12(5): 950-965.
[5] Yong WANG, Zhi-Zhong LIU, Jianbin LI, Han-Xiong LI, Jiahai WANG. On the selection of solutions for mutation in differential evolution[J]. Front. Comput. Sci., 2018, 12(2): 297-315.
[6] Jayanthi MANICASSAMY, Dinesh KARUNANIDHI, Sujatha POTHULA, Vengattaraman THIRUMAL, Dhavachelvan PONNURANGAM, Subramanian RAMALINGAM. GPS: a constraint-based gene position procurement in chromosome for solving large-scale multiobjective multiple knapsack problems[J]. Front. Comput. Sci., 2018, 12(1): 101-121.
[7] Ruochen LIU,Chenlin MA,Fei HE,Wenping MA,Licheng JIAO. Reference direction based immune clone algorithm for many-objective optimization[J]. Front. Comput. Sci., 2014, 8(4): 642-655.
[8] Yongjian ZHAO, Hong HE, Jianxun Mi. Noisy component extraction with reference[J]. Front Comput Sci, 2013, 7(1): 135-144.
[9] Tim SCHLüTER, Stefan CONRAD. An approach for automatic sleep stage scoring and apnea-hypopnea detection[J]. Front Comput Sci, 2012, 6(2): 230-241.
[10] Hanyang QUEK, Chunghoong WOO, Kaychen TAN, Arthur TAY. Evolving Nash-optimal poker strategies using evolutionary computation[J]. Front Comput Sci Chin, 2009, 3(1): 73-91.
[11] Maryjane TREMAYNE, Samantha Y. CHONG, Duncan BELL. Optimisation of algorithm control parameters in cultural differential evolution applied to molecular crystallography[J]. Front Comput Sci Chin, 2009, 3(1): 101-108.
[12] Carlos A. COELLO COELLO. Evolutionary multi-objective optimization:some current research trends and topics that remain to be explored[J]. Front Comput Sci Chin, 2009, 3(1): 18-30.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed