Journal cover Journal topic
Hydrology and Earth System Sciences An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 5.153 IF 5.153
  • IF 5-year value: 5.460 IF 5-year
    5.460
  • CiteScore value: 7.8 CiteScore
    7.8
  • SNIP value: 1.623 SNIP 1.623
  • IPP value: 4.91 IPP 4.91
  • SJR value: 2.092 SJR 2.092
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 123 Scimago H
    index 123
  • h5-index value: 65 h5-index 65
Preprints
https://doi.org/10.5194/hess-2020-237
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/hess-2020-237
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Submitted as: technical note 12 Jun 2020

Submitted as: technical note | 12 Jun 2020

Review status
This preprint is currently under review for the journal HESS.

Technical note: Diagnostic efficiency – specific evaluation of model performance

Robin Schwemmle, Dominic Demand, and Markus Weiler Robin Schwemmle et al.
  • University of Freiburg, Faculty of Environment and Natural Resources, Chair of Hydrology, Freiburg, Germany

Abstract. Better understanding of the reasons why hydrological model performance is good or poor represents a crucial part for meaningful model evaluation. However, current evaluation efforts are mostly based on aggregated efficiency measures such as Kling-Gupta Efficiency (KGE) or Nash-Sutcliffe Efficiency (NSE). These aggregated measures only distinguish between good and poor model performance. Especially in the case of a poor model performance it is important to identify the different errors which may have caused such unsatisfying predictions. These errors may origin from the model parameters, the model structure, and/or the input data. In order to provide more insight, we define three types of errors which may be related to their origin: constant error (e.g. caused by consistent input data error such as precipitation), dynamic error (e.g. structural model errors such as a deficient storage routine) and timing error (e.g. caused by input data errors or deficient model routines/parameters). Based on these types of errors, we propose the novel Diagnostic Efficiency (DE) measure, which accounts for the three error types. The disaggregation of DE into its three metric terms can be visualized in a plain radial space using diagnostic polar plots. A major advantage of this visualization technique is that error contributions can be clearly differentiated. In order to provide a proof of concept, we first generated errors systematically by mimicking the three error types (i.e. simulations are surrogated by manipulating observations). By computing DE and the related diagnostic polar plots for the mimicked errors, we could then supply evidence for the concept. Finally, we tested the applicability of our approach for a modelling example. For a particular catchment, we compared streamflow simulations realized with different parameter sets to the observed streamflow. For this modelling example, the diagnostic polar plot suggests, that dynamic errors explain the model performance to a large extent. The proposed evaluation approach provides a diagnostic tool for model developers and model users and the diagnostic polar plot facilitates interpretation of the proposed performance measure.

Robin Schwemmle et al.

Interactive discussion

Status: open (until 07 Aug 2020)
Status: open (until 07 Aug 2020)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement

Robin Schwemmle et al.

Model code and software

diag-eff 0.1: Diagnostic efficiency – specific evaluation of model performance R. Schwemmle, D. Demand, and M. Weiler https://doi.org/10.5281/zenodo.3769562

Robin Schwemmle et al.

Viewed

Total article views: 369 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
267 101 1 369 27 2 1
  • HTML: 267
  • PDF: 101
  • XML: 1
  • Total: 369
  • Supplement: 27
  • BibTeX: 2
  • EndNote: 1
Views and downloads (calculated since 12 Jun 2020)
Cumulative views and downloads (calculated since 12 Jun 2020)

Viewed (geographical distribution)

Total article views: 305 (including HTML, PDF, and XML) Thereof 305 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 

Cited

Saved

No saved metrics found.

Discussed

No discussed metrics found.
Latest update: 13 Jul 2020
Publications Copernicus
Download
Short summary
Better understanding of the reasons why model performance is good or poor represents a crucial part for meaningful model evaluation. We propose the novel Diagnostic Efficiency (DE) measure and diagnostic polar plots. The proposed evaluation approach provides a diagnostic tool for model developers and model users and facilitates interpretation of model performance.
Better understanding of the reasons why model performance is good or poor represents a crucial...
Citation