Alewell, C; Manderscheid, B: Use of objective criteria for the assessment of biogeochemical ecosystem models, Ecological Modelling, 107, 213-224 (1998)
Abstract: Ecosystem modeling is confronted with complex biological systems and changing environmental conditions. A model which describes ecosystem behavior under all conditions has not been found yet and there does not exist one ‘true’ model for a specific ecosystem. Often ecosystem models describe the measured data more or less well, but most judging criteria for model performance are rather subjective. Furthermore, from a mathematical view point the calibrations of ecosystem models are hardly ever unique.
The aim of this study was to develop and use criteria which permit an objective comparison of different models to the observed field data and to each other. A given model which describes a specific system significantly better will be declared the ‘valid’ model while the other will be rejected. The term "valid" is used here in a sense that any model that could not be proven invalid would be a valid model for the system.
We used the biogeochemical soil models MAGIC (Cosby et al., 1985) and the SO-Model (derived from the Batch Equilibrium Model (BEM), Prenzel, 1991). The data set used was the soil solution chemistry in a forest ecosystem of the Solling area (North-West Germany). To test the performance of the models four criteria were used: the efficiency (Martinec and Rango, 1989; Hinzman and Kane, 1991), the Normalized Mean Absolute Error (NMAE, given by Janssen and Heuberger, 1995), the confidence interval test (CIT, developed in this study) and the model rejection criteria (Sun, 1994). Whereas the efficiency and NMAE are related to the averaged data, the CIT and the model rejection criteria include the spatial heterogeneity at every time step.
When evaluated visually, both model results might be accepted. From the application of the model performance criteria we selected the MAGIC model as the ‘valid’ model for our system.