Quality control of specific humidity from surface stations based on EOF and FFT—Case study

doi:10.1007/s11707-014-0483-2

Front. Earth Sci.

2015, Vol. 9

Issue (3) : 381-393 https://doi.org/10.1007/s11707-014-0483-2

RESEARCH ARTICLE

Quality control of specific humidity from surface stations based on EOF and FFT—Case study

Hong ZHAO¹,Xiaolei ZOU^2,^*(

),Zhengkun QIN¹

1. Center of Data Assimilation for Research and Application, Nanjing University of Information Science & Technology, Nanjing 210044, China
2. Earth System Science Interdisciplinary Center, University of Maryland, College Park, MD 20740-3823, USA

Download: PDF(2727 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Comparisons between observations and background fields indicate that amplitude and phase differences in oscillations result in a non-Gaussian distribution in observation minus background vectors (OMB). Empirical Orthogonal Function (EOF) quality control (QC) and Fast Fourier Transform (FFT) quality control are proposed from the perspective of data assimilation and are applied to the surface specific humidity from ground-based stations. The QC results indicate that the standard deviation between observations and background is reduced effectively, and the frequency distribution for the observation increment is closer to a normal distribution. The specific humidity outliers occur primarily in mountainous and coastal regions. Comparing the two QC methods, it is found that the EOF QC performs better than the FFT QC as it can keep large scale of fluctuation information from the original field, preventing these waves from entering into the residual field and being removed by the QC process.

Keywords specific humidity quality control EOF FFT

Corresponding Author(s): Xiaolei ZOU

Just Accepted Date: 26 September 2014 Issue Date: 20 July 2015

Cite this article:

Xiaolei ZOU,Zhengkun QIN,Hong ZHAO. Quality control of specific humidity from surface stations based on EOF and FFT—Case study[J]. Front. Earth Sci., 2015, 9(3): 381-393.

URL:

https://academic.hep.com.cn/fesci/EN/10.1007/s11707-014-0483-2
https://academic.hep.com.cn/fesci/EN/Y2015/V9/I3/381

Fig.1 Power spectrum density of surface specific humidity from observations (a) and background field (b), in 55°E?145°E, 0?70°N. The 95% confidence level is shown (dashed line). Data are from the first eight days of January 2008.

Fig.2 Cumulative variances explained by the first 15 EOF modes for specific humidity from observations (shaded) and background fields (dashed line) within an 8-day period ending at different observing times indicated by the numbers on the x-axis. The time of 0000 UTC 9 January 2008 is the 65th observing time in January 2008.

Fig.3 PCs of the first ten EOF modes of specific humidity from observations (solid line) and background fields (dashed line) from 0000 UTC 1 January to 0000 UTC 8 January 2008. The number indicates the EOF mode number.

Fig.4 Modulus of wavelet analysis for the PCs of the first ten EOF modes of specific humidity from observations (a, c) and background fields (b, d) for oscillation periods with 18?72 h (top panels a and b) and 6?18 h (bottom panels c and d).

Fig.5 Frequency distributions of the differences between the observations and background fields for the EOF rebuild terms q E O F r e b O B S - q E O F r e b B F (a), and the composite fields extracted from FFT q F F T r e b O B S - q F F T r e b B F (b). The solid curve is the Gaussian fitting.

Fig.6 The spatial distributions of standard deviations for the EOF residual terms (a) and FFT residual terms (b) of specific humidity.

Fig.7 The bi-weighted standard deviations of q E O F r e s O B S - q E O F r e s B F (solid lines) and q F F T r e s O B S - q F F T r e s B F (dashed lines) in the north region (thick lines) and the south region (thin lines) (See Fig. 6 for the two regions).

Fig.8 Frequency distributions of the differences of q E O F r e s O B S - q E O F r e s B F (a, b) and q F F T r e s O B S - q F F T r e s B F (c, d) before (white bar) and after (shaded bar) QC. For clarity, only the tails of the frequency distributions are shown.

Tab.1 Mean, Standard Deviation, Skewness, and Kurtosis of OMB of variables q O B S - q B F , q E O F r e b O B S - q E O F r e b B F , q F F T r e b O B S - q F F T r e b B F , q E O F res O B S - q E O F r e s B F , q F F T r e s O B S - q F F T r e s B F , before and after QC.

Fig.9 Total number of specific humidity data removed at each surface station by (a) EOF QC, (b) FFT QC and (c) both QCs.

Fig.10 The percentage of surface specific humidity removed at each observing time by QC methods using the EOF and FFT methods.

Fig.11 Spatial distributions of the outliers identified by only the EOF QC (blue points), only the FFT QC (red points), and by both methods (green points) at 1500 UTC 10 January 2008.

Fig.12 Spatial distributions of (a) q , (b) q F F T r e b , (c) q E O F r e b , (d) q F F T r e s and (e) q E O F r e s from observations (solid lines), FNL data (dashed lines), and the differences between observations and FNL data (shaded areas: q O B S - q B F in (a), q F F T r e b O B S - q F F T r e b B F in (b), q E O F r e b O B S - q E O F r e b B F in (c), q F F T r e s O B S - q F F T r e s B F in (d), and q E O F res O B S - q E O F r e s B F in (e)) at 1500 UTC 10 January 2008 in the subdomain indicated by the small box in Fig. 11.

1	Anderson E, J?rvinen H (1999). Variational quality control. Q J R Meteorol Soc, 125(554): 697–722 https://doi.org/10.1002/qj.49712555416
2	Baker N L (1992). Quality control for the Navy operational atmospheric database. Weather Forecast, 7(2): 250–261 https://doi.org/10.1175/1520-0434(1992)007<0250:QCFTNO>2.0.CO;2
3	Chen Q Y, Guan C G, Yao M M, Tong H, Mariano H, Li Z C (2007). Development of key techniques and experiments in global model upgrading. Acta Meteorologica Sinica, 65(4): 478–492 (in Chinese)
4	Cooley J W, Tukey O W (1965). An algorithm for the machine calculation of complex Fourier series. Math Comput, 19(90): 297–301 https://doi.org/10.1090/S0025-5718-1965-0178586-1
5	Eischeid J K, Bruce Baker C, Karl T R, Diaz H F (1995). The quality control of long-term climatological data using objective data analysis. J Appl Meteorol, 34(12): 2787–2795 https://doi.org/10.1175/1520-0450(1995)034<2787:TQCOLT>2.0.CO;2
6	Fan S Y, Zhang C L (2006). Numerical assessing experiments on individual component impact of the meteorological observation network on the “July 2000” torrential rain in Beijing. Acta Meteorologica Sinica, 20(4): 389–401 (in Chinese)
7	Feng S, Hu Q, Qian W (2004). Quality control of daily meteorological data in China, 1951–2000: a new dataset. Int J Climatol, 24(7): 853–870 https://doi.org/10.1002/joc.1047
8	Gandin L S (1988). Complex quality control of meteorological observations. Mon Weather Rev, 116(5): 1137–1156 https://doi.org/10.1175/1520-0493(1988)116<1137:CQCOMO>2.0.CO;2
9	Ghil M, Malanotte-Rizzoli P (1991). Data assimilation in meteorology and oceanography. Adv Geophys, 33: 151?156 https://doi.org/10.1016/S0065-2687(08)60442-2
10	Guo Y R, Shin D H, Lee J H, Xiao Q N, Barker D M, Kuo Y H (2002). Application of the MM5 3DVAR system for a heavy rain case over the Korean Peninsula. In: Proccedings: the Twelfth PSU/NCAR Mesoscale Model Users’ Workshop NCAR, Boulder, 24–25
11	Ha K J, Jeon E H, Oh H M (2007). Spatial and temporal characteristics of precipitation using an extensive network of ground gauge in the Korean Peninsula. Atmos Res, 86(3–4): 330–339 https://doi.org/10.1016/j.atmosres.2007.07.002
12	Hong S Y, Pan H L (1996). Nonlocal boundary layer vertical diffusion in a medium-range forecast model. Mon Weather Rev, 124(10): 2322–2339 https://doi.org/10.1175/1520-0493(1996)124<2322:NBLVDI>2.0.CO;2
13	Hubbard K G, You J (2005). Sensitivity analysis of quality assurance using the spatial regression approach—A case study of the maximum/minimum air temperature. J Atmos Ocean Technol, 22(10): 1520–1530 https://doi.org/10.1175/JTECH1790.1
14	Ingleby N B, Lorenc A C (1993). Bayesian quality control using multivariate normal distributions. Q J R Meteorol Soc, 119(513): 1195–1225 https://doi.org/10.1002/qj.49711951316
15	Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woolen J, Zhu Y, Leetmaa A, Reynolds R, Chelliah M, Ebisuzaki W, Higgins W, Janowiak J, Mo K C, Ropelewski C, Wang J, Jenne R, Joseph D (1996). The NCEP/NCAR 40-year reanalysis project. Bull Am Meteorol Soc, 77(3): 437–471 https://doi.org/10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2
16	Lanzante J R (1996). Resistant, robust and non-parametric techniques for the analysis of climate data: theory and examples, including applications to historical radiosonde station data. Int J Climatol, 16(11): 1197–1226 https://doi.org/10.1002/(SICI)1097-0088(199611)16:11<1197::AID-JOC89>3.0.CO;2-L
17	Li L, Njoku E G, Im E, Chang P S, Germain K S (2004). A preliminary survey of radio-frequency interference over the US in Aqua AMSR-E data. Geoscience and Remote Sensing. IEEE Transactions on, 42(2): 380–390
18	Lorenc A C, Hammon O (1988). Objective quality control of observations using Bayesian methods. Theory, and a practical implementation. Q J R Meteorol Soc, 114(480): 515–543 https://doi.org/10.1002/qj.49711448012
19	Lorenz E N (1956). Empirical orthogonal functions and statistical weather prediction, scientific Report No. 1, Statistical Forecasting Project. Massachusetts Institute of Technology department of meteorology
20	Mardia K V (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3): 519–530 https://doi.org/10.1093/biomet/57.3.519
21	Mohanty U C, Kasahara A, Errico R (1986). The impact of diabatic heating on the initialization of a global forecast model. J Meteorol Soc Jpn, 64(6): 805–817
22	Onogi K, Tsutsui J, Koide H, Sakamoto M, Kobayashi S, Hatsushika H, Matsumoto T, Yamazaki N, Kamahori H, Takahashi K, Kadokura S, Wada K, Kato K, Oyama R, Ose T, Mannoji N, Taira R (2007). The JRA-25 reanalysis. J Meteorol Soc Jpn, 85(3): 369–432 https://doi.org/10.2151/jmsj.85.369
23	Qin Z K, Zou X L, Li G, Ma X L (2010). Quality control of surface station temperature data with non-Gaussian observation-minus-background distributions. J Geophys Res, 115(D16): D16312 https://doi.org/10.1029/2009JD013695
24	Ramanathan V, Barkstrom B R, Harrison E F (1989). Climate and the earth’s radiation budget. Phys Today, 42(5): 22–32 https://doi.org/10.1063/1.881167
25	Reek T, Doty S R, Owen T W (1992). A deterministic approach to the validation of historical daily temperature and precipitation data from the cooperative network. Bull Am Meteorol Soc, 73(6): 753–762 https://doi.org/10.1175/1520-0477(1992)073<0753:ADATTV>2.0.CO;2
26	Sasaki Y (1970). Some basic formalisms in numerical variational analysis. Mon Weather Rev, 98(12): 875–883 https://doi.org/10.1175/1520-0493(1970)098<0875:SBFINV>2.3.CO;2
27	Shafer M A, Fiebrich C A, Arndt D S, , Fredrickson S E, Hughes T W (2000). Quality assurance procedures in the Oklahoma Mesonetwork. J Atmos Ocean Technol, 17(4): 474–494 https://doi.org/10.1175/1520-0426(2000)017<0474:QAPITO>2.0.CO;2
28	Simmons A, Uppala S, Dee D, Kobayashi S (2007). ERA-Interim: new ECMWF reanalysis products from 1989 onwards. ECMWF newsletter, 110(110): 25–35
29	Torrence C, Compo G P (1998). A practical guide to wavelet analysis. Bull Am Meteorol Soc, 79(1): 61–78 https://doi.org/10.1175/1520-0477(1998)079<0061:APGTWA>2.0.CO;2
30	Wade C G (1987). A quality control program for surface mesometeorological data. J Atmos Ocean Technol, 4(3): 435–453 https://doi.org/10.1175/1520-0426(1987)004<0435:AQCPFS>2.0.CO;2
31	Wang Y, Xu Z F, Fan G Z (2013). Research of quality control based on EOF for 2m temperature. Plateau Meteorology, 32(2): 564–574 (in Chinese)
32	Wei F Y (2007). Statisticial Diagnosis and Prediction for Modern Climate. Beijing: China Meteorological Press, 71–76 (in Chinese)
33	Zhao J, Zou X L, Weng F Z (2013). WindSat radio-frequency interference signature and its identification over Greenland and Antarctic. IEEE Trans Geosci Rem Sens, 51(9): 4830–4839 https://doi.org/10.1109/TGRS.2012.2230634
34	Zou X L (2009). Theories and Applications for Data Assimilation (I). Beijing: China Meteorological Press (in Chinese)
35	Zou X L, Ma Y, Qin Z K (2012). Fengyun-3B microwave humidity sounder (MWHS) data noise characterization and filtering using principle component analysis. IEEE Trans Geosci Rem Sens, 50(12): 4892–4902 https://doi.org/10.1109/TGRS.2012.2202122
36	Zou X L, Qin Z K (2010). Time zone dependence of diurnal cycle errors in surface temperature analyses. Mon Weather Rev, 138(6): 2469–2475 https://doi.org/10.1175/2010MWR3248.1
37	Zou X L, Zeng Z (2006). A quality control procedure for GPS radio occultation data. J Geophys Res, 111: D02112 https://doi.org/10.1029/2005JD005846

[1]	Yin LIU,Xiaolei ZOU. Quality control of AIRS total column ozone data within tropical cyclones[J]. Front. Earth Sci., 2016, 10(2): 222-235.
[2]	Juan LI,Guiqing LIU. Assimilation of Chinese Fengyun-3B Microwave Temperature Sounder radiances into the Global GRAPES system with an improved cloud detection threshold[J]. Front. Earth Sci., 2016, 10(1): 145-158.
[3]	Tianyong ZHAI,Qing ZHAO,Wei GAO,Runhe SHI,Weining XIANG,Hung-lung Allen HUANG,Chao ZHANG. Analysis of spatio-temporal variability of aerosol optical depth with empirical orthogonal functions in the Changjiang River Delta, China[J]. Front. Earth Sci., 2015, 9(1): 1-12.
[4]	Dan JIAO, Huan YANG, Xinjun WANG, Shucheng XIE, Shuyuan XIANG, . Paleofire indicated by triterpenes and charcoal in a culture bed in eastern Kunlun Mountain, Northwest China[J]. Front. Earth Sci., 2009, 3(4): 452-456.

Viewed

Full text

Abstract

Cited

Shared

Discussed