KnowEarth: Machine learning and human knowledge

How can we integrate human knowledge with modern deep learning models?

Earth observation (EO) systems offer unprecedented volumes of data. To utilise this information, automated analysis of EO data is an ongoing research activity. Deep learning, a popular type of AI model, has revolutionised this work and is currently the leading tool for solving a wide range of tasks pertaining to remote sensing and image analysis.

Modern AI models are highly data driven and are often trained on thousands of annotated data. By annotated data, we refer to data where the content of each sample is known. The models are powerful tools, but often lack commonsense knowledge and an understandig of basic principles that govern the real world. Consequently, this results in predictions that often contradict well-known rules and constraints.

Enhancing today’s deep learning models

Our objective is to advance deep learning models by exploiting and integrating human knowledge, constraints, and physics into the models. The models will subsequently be used to significantly elevate automatic analysis of EO image data. This will make the training make the training of deep learning models more efficient, and means that will we need a smaller dataset to train the models precisely. Furthermore, it paves the way for applying unlabeled data when training the deep learning models of the future.

Improved monitoring of the Earth

The project focuses on two EO challenges: the mapping of wetlands and the mapping of snow cover from satellite image data. We anticipate significant methodical improvement of land cover mapping which, in turn, will contribute to better management of wetland ecosystems. Enhanced snow cover maps will improve estimates of water resources available for hydropower production, in addition to bettering climate change monitoring in areas with snow, ice and permafrost.