Volume 11, Issue 2 (8-2024)
Journal of Spatial Analysis Environmental Hazards 2024, 11(2): 117-136 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Mirshafie A. Assessment of the measurement statistics of model accuracy and the appropriate use of them (Case study: Interpolation of Precipitation in Fars province). Journal of Spatial Analysis Environmental Hazards 2024; 11 (2) : 7
URL: http://jsaeh.khu.ac.ir/article-1-3401-en.html
URL: http://jsaeh.khu.ac.ir/article-1-3401-en.html
Abstract: (2191 Views)
Assessment of the measurement statistics of model accuracy and the appropriate
use of them (Case study: Interpolation of Precipitation in Fars province)
Abstract
In many scientific researches, error measurement statistics are often used without taking notices into account
when selecting a model or method for the spatial analysis of environmental hazards. In order to assess the
accuracy of precipitation interpolation methods in Fars province, the performance of widely used error
measurement statistics and some comments were implemented. Spatial interpolation of precipitation was
accomplished using inverse distance weighting, kriging, co-kriging, and radial basis functions methods with 161
weather stations (22 synoptic and 139 rain gauge stations) for 2018 as a rainy year. The results of MBE statistic
evaluation indicated that the researcher may have chosen the incorrect interpolation method in certain cases
where the sum of the positive and negative values became zero. In addition, this statistic is limited to indicating
overestimation or underestimation and should not be used for assessing accuracy or selecting interpolation
techniques. Regarding the coefficient of determination (r 2 ), the results revealed that due to the lack of
compatibility in the magnitude of the range of this coefficient (0 to 1) with error values (100 to 400 mm for the
interpolation of precipitation in Fars province), its use in evaluation of the accuracy of a method is not
recommended. In terms of NRMSE, the results showed that samples with a small number of observations (n=3),
its value increased excessively (NRMSE=0.35) when compared to samples with a bigger number of data (n=20,
NRMSE=0.097). Therefore, it is not advised to use this statistic. In conclusion, since MAE and RMSE statistics
provide a more realistic error value, it is advised to use them for assessing the accuracy of interpolation
methods.
Keywords: Precipitation, Error evaluation statistics, Interpolation methods, Fars province
Article number: 7
Type of Study: Research |
Subject:
Special
Received: 2023/11/18 | Accepted: 2024/09/14 | Published: 2024/09/14
Received: 2023/11/18 | Accepted: 2024/09/14 | Published: 2024/09/14
References
1. اصغری‌مقدم، اصغر؛ وحید نورانی و عطاالله ندیری. 1388. پیش بینی زمانی و مکانی سطح آب‌های زیرزمینی در محدوده متروی شهر تبریز با استفاده از مدل کریجینگ عصبی، تحقیقات منابع آب ایران، 13: 14-24.
2. حسنی پاک، علی اصغر؛ محمد شرف الدین. 1390.تحلیل داده‌های اکتشافی، مؤسسه انتشارات دانشگاه تهران.
3. حسینی، سیده فاطمه؛ همتی، محمد؛ جعفری، مهتاب؛ استعلاجی، علیرضا. 1402. تحلیل و پهنهبندی خطر سیلخیزی و ارتباط آن با پوشش گیاهی در شهرستان قیروکارزین، تحلیل فضایی مخاطرات محیطی، 10(2): 77-96.
4. خسروی، علیرضا؛ اژدری مقدم، مهدی؛ هاشمیفرد، سید آرمان؛ نظری پور، حمید. 1401. مقایسه نتایج تصمیمگیری چند معیاره در پهنهبندی مناطق مستعد خطر سیلاب با شاخصهای سنجش از دور در حوضه آبریز رودخانه کهیر (بلوچستان جنوبی)، تحلیل فضایی مخاطرات محیطی، 9(4): 21-40.
5. زندکریمی، آرش؛ داود مختاری. 1397. ارزیابی دقت روش‌های مختلف درون‌یابی در تخمین مقادیر بارش جهت انتخاب بهینه‌ترین الگوریتم در استان کردستان، پژوهش‌های جغرافیایی طبیعی، دوره 5، شماره2: 338-323.
6. شمسی‌پور، علی اکبر. 1393. مدلسازی آب وهوایی، انتشارات دانشگاه تهران.
7. عیوضی، معصومه؛ مشاعدی، ابوالفضل. 1390پایش و تحلیل مکانی خشکسالی هواشناسی در سطح استان گلستان با استفاده از روش‌های زمینآماری، مرتع و آبخیزداری، 64(1):65-78.
8. مرادی، اسحاق؛ افسانه شهبازی؛ کاظم نصرتی؛ غلامرضا زهتابیان. 1385. ارزیابی روش شبیه‌سازی تصادفی برای تولید داده‌های هواشناسی، پژوهش‌های جغرافیایی- شماره 62: 1-9.
9. ندیری، عطاالله؛ صدیقه شکور. 1393. ارزیابی انواع روش‌های درون‌یابی، جهت تخمین آلودگی نیترات در منابع آب زیرزمینی ، مجله هیدروژئومورفولوژی، شماره 1: 92-75.
10. Alimissis, A.; k. Philippopoulos, C. Gtzanis, and D. Deligiorgi. 2018. Spatial estimation of urban air pollution with the use of artificial neural network models. Atmospheric Environment. 191: 205-213.
11. Belkhiri L.; A. Tiri, and L. Mouni. 2020. Spatial distribution of the groundwater quality using kriging and Co-kriging interpolations. Groundwater for Sustainable Development. 11: 100-73.
12. Chai, T.; And R. Draxler. 2004. root mean square (RMSE) or mean absolute error (MAE) Arguments against avoiding RMSE in the literature. Geosci model Dev. 7: 1247-1250.
13. Davies, j.; M. Abdel-wahab, and D. Makay.1984. nating solar iradiation on horizontal Surfaces. solar Energy. 32: 307-309.
14. Ding, Q.; Y. Wang, and D. Zhuang. 2018. Comparison of the common spatial interpolation methods used to analyze potentially toxic elements surrounding mining regions. Journal of Environmental Management. 212: 23-31.
15. García-Santos, G.; M. Scheiber, and J. Pilz. 2020. Spatial interpolation methods to predict airborne pesticide drift deposits on soils using knapsack sprayers. Chemosphere. 258: 127-231.
16. Falivene, O.; R. Cabrera, R. Tolosana-Delgado, and A. Saez. 2010. Interpolation algorithm ranking Using cross-validation and the role of asmoothing effect:A coal zone example Comput.Geosci, 36: 512-519.
17. Fekete, B.; C. Vörösmarty, J. Roads, and C. Willmott .2004. Uncertainties in precipitation and their impacts on runoff estimates. Journal of Clim. 17: 294–304
18. Fan, J.; and I. Gibels. 1996. Local Polynomial Modelling and Its Applications. Water Resources Bulletin. 87: 998-1004.
19. Hyndman, R.; and A. Koehler. 2005. Another look at measures of forecast accuracy . International Journal of Forecasting. 22: 679-688
20. Li, J.; H. Wan, and S. Shang. 2020. Comparison of interpolation methods for mapping layered soil particle-size fractions and texture in an arid oasis. Catena. 190: 104-1014.
21. Jacovides, C.; G. Papaioannou, and P. Kerkides. 1994. Micro and large-scale parameters evaluation of evaporation from a lake. Agricultural Water Management. 13: 263-27
22. Kazemi, S.; and S. Hosseini. 2011. Comparison of spatial interpolation methods for estimating heavy metals in sediments of Caspian Sea. Expert Systems with Applications. 38: 1632-1649.
23. Nekoamal, M.; and R. Mirabbasi. 2017. Assessment of interpolation methods in estimation of groundwater level (case study: sarkhon plain). Journal of hydrogeology, online publish. 2: 84-95.
24. Nash, J.; and E. Sutcliffe. 1970. River Flow forecasting through conceptual models, part 1-A discussion of principles. Journal of Hydrology, 10: 282-290.
25. Salah, H.; 2009. Geostatistical analysis of groundwater levels in the south Al Jabal Al Akhdar area using GIS. GIS Ostrava. 25: 1-10.
26. Willmott, C.; and K. Matsuura. 2005. advantages of the mean Absolute Error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research. 30: 79-82.
27. Xin, Y.; and G, Xiao 2009. Linear regression analysis: theory and computing. world Scientific Publishin. 348.
28. Yuval-Levy, I.; and D, Broday. 2017. Improving modeled air pollution concentration maps by residual interpolation. Science of The Total Environment. 598: 780-788.
29. Alimissis, A.; k. Philippopoulos, C. Gtzanis, and D. Deligiorgi. 2018. Spatial estimation of urban air pollution with the use of artificial neural network models. Atmospheric Environment. 191: 205-213.
30. Asghari Moghadam, A.; V. Norani, and A. O. Nadiri. 2009. Forecasting Spatiotemporal Water Levels by Neural Kriging Method in Tabriz City Underground Area. Iran Water Resources Research. 191: 205-213. (In Persian)
31. Belkhiri L.; A. Tiri, and L. Mouni. 2020. Spatial distribution of the groundwater quality using kriging and Co-kriging interpolations. Groundwater for Sustainable Development. 11: 100-73.
32. Chai, T.; And R. Draxler. 2004. root mean square (RMSE) or mean absolute error (MAE) Arguments against avoiding RMSE in the literature. Geosci model Dev. 7: 1247-1250.
33. Davies, j.; M. Abdel-wahab, and D. Makay.1984. nating solar iradiation on horizontal Surfaces. solar Energy. 32: 307-309.
34. Ding, Q.; Y. Wang, and D. Zhuang. 2018. Comparison of the common spatial interpolation methods used to analyze potentially toxic elements surrounding mining regions. Journal of Environmental Management. 212: 23-31.
35. García-Santos, G.; M. Scheiber, and J. Pilz. 2020. Spatial interpolation methods to predict airborne pesticide drift deposits on soils using knapsack sprayers. Chemosphere. 258: 127-231.
36. Falivene, O.; R. Cabrera, R. Tolosana-Delgado, and A. Saez. 2010. Interpolation algorithm ranking Using cross-validation and the role of a smoothing effect:A coal zone example Comput.Geosci, 36: 512-519.
37. Fekete, B.; C. Vörösmarty, J. Roads, and C. Willmott .2004. Uncertainties in precipitation and their impacts on runoff estimates. Journal of Clim. 17: 294–304
38. Fan, J.; and I. Gibels. 1996. Local Polynomial Modelling and Its Applications. Water Resources Bulletin. 87: 998-1004.
39. Hasani Pak, A. A. and M. Sharafaldin. 2011. Expolratory Data Analisis. University of Tehran. (In Persian)
40. Hyndman, R.; and A. Koehler. 2005. Another look at measures of forecast accuracy. International Journal of Forecasting. 22: 679-688
41. Li, J.; H. Wan, and S. Shang. 2020. Comparison of interpolation methods for mapping layered soil particle-size fractions and texture in an arid oasis. Catena. 190: 104-1014.
42. Jacovides, C.; G. Papaioannou, and P. Kerkides. 1994. Micro and large-scale parameters evaluation of evaporation from a lake. Agricultural Water Management. 13: 263-27
43. Kazemi, S.; and S. Hosseini. 2011. Comparison of spatial interpolation methods for estimating heavy metals in sediments of Caspian Sea. Expert Systems with Applications. 38: 1632-1649.
44. Moradi, I. A. ; Shahbazi; K. Nosrati and Gh. Zehtabian. 2006. Assessment of Random Simulation Method for Producing Meteorological Data. Physical Geography Research Quarterly. 62: 1-9. (In Persian)
45. Nadiri, A. ; S. Shakoor; A. Asghari Moghadam And M. Vadiati. Investigation of Groundwater Nitrate Pollution with Different Interpolation Methods (Case Study: East Azarbayjan, Bilverdy Plain). 2015. Hydrogeomorphology. 1: 75-92. (In Persian)
46. Nekoamal, M.; and R. Mirabbasi. 2017. Assessment of interpolation methods in estimation of groundwater level (case study: Sarkhon plain). Journal of hydrogeology, online publishing. 2: 84-95.
47. Nash, J.; and E. Sutcliffe. 1970. River Flow forecasting through conceptual models, part 1-A discussion of principles. Journal of Hydrology, 10: 282-290.
48. Salah, H.; 2009. Geostatistical analysis of groundwater levels in the south Al Jabal Al Akhdar area using GIS. GIS Ostrava. 25: 1-10.
49. Shamsipour, A. A. 2014. Climate Modeling. University of Tehran. (In Persian)
50. Willmott, C.; and K. Matsuura. 2005. advantages of the mean Absolute Error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research. 30: 79-82.
51. Xin, Y.; and G, Xiao 2009. Linear regression analysis: theory and computing. world Scientific Publishing. 348.
52. Yuval-Levy, I.; and D, Broday. 2017. Improving modeled air pollution concentration maps by residual interpolation. Science of The Total Environment. 598: 780-788.
53. Zandkarimi, A. and D. Mokhtari. 2018. Accuracy of Various Interpolation Methods in Estimating Rainfall Values to Select the Most Optimal Interpolation Algorithm (Case Study: Kurdistan province. Physical Geography Research Quarterly. 5: 323-338. (In Persian)
| Rights and permissions | |
 | This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
