Knowledge-guided machine learning – a practical handbook for ecologists

Interest in advanced machine learning, particularly deep learning and artificial intelligence algorithms has grown within ecological research in recent years. This growth has largely been driven by the effectiveness of the resulting models for representing complex non-linear relationships and interactions in high-dimensional datasets. Despite these advances, adoption of deep learning over conventional modeling tools remains limited by three critical challenges: (1) the scarcity of large, representative training datasets; (2) the inherent lack of mechanistic interpretability in the resulting "black box" models, which obscure how predictions are made; and (3) poor generalization to novel environmental conditions, such as those altered by climate change. However, these challenges can at least be partially overcome by integrating domain-specific knowledge into a machine learning framework—an approach referred to in other fields as knowledge-guided machine learning (KGML). KGML incorporates established scientific knowledge, such as conservation laws or governing equations, as “soft” (guiding) or “hard” (enforced) constraints during model training and architecture design. This approach has several benefits over the naïve application of deep learning algorithms (“naïve” meaning purely data-driven). First, by incorporating prior knowledge, KGML can narrow the parameter search space to enable effective learning from smaller datasets. Second, KGML can improve mechanistic interpretability through modular architectures that separate knowledge- and data-driven components. Lastly, KGML can improve generalization by ensuring that model outputs are consistent with built-in scientific knowledge. While KGML approaches are well-established in other areas of science since several years, their adoption in ecological modeling and global change research is still in the early stages. Therefore, our goal here is to (1) introduce these methods to a broader audience within ecological modeling and (2) offer a practical handbook for implementing these methods.