A Supervised Machine Learning approach to foliage temperature extraction from natural environments using thermal imagery

Sean Carpenter
Graduate (MS)
Darren Drewry
Food, Agricultural and Biological Engineering

According to the United States Department of Agriculture, projections show that food production needs to increase by 70-100% from 2010 – 2050 due to population growth in addition to other socioeconomic pressures. New methods are needed to increase the productivity and efficiency of agricultural systems and are critical for mitigating climate change and ensuring food security. Remote sensing (RS) and Unmanned Aerial Systems (UAS) have the potential to allow agricultural researchers to better manage and monitor these complex systems. Foliage temperature is a key variable in biophysical vegetative modeling and has been well documented to be an indicator of crop water stress. The ability to monitor subtle changes in foliage temperature using a calibrated thermal infrared (TIR) camera mounted on a UAS would open avenues for field-based stress monitoring at scales not possible without using airborne systems. However, current approaches to process thermal image data are time-consuming, inaccurate, or not well suited for foliage in field environments. And importantly, methods to extract foliage pixels from the background (i.e. soil, weeds, etc) are needed to remove the influence of background elements that can have dramatically different temperatures from the surrounding plant tissue. This study aims to train and validate a Supervised Machine Learning (SML) algorithm using a dual-camera system to extract foliage temperature in a complex field environment. A UAS campaign focused on a set of maize treatments was conducted at Waterman Farm throughout the summer of 2020, spanning diurnal acquisitions across the growing season. In-situ tower-based sensors were deployed to provide validation of the airborne data. Remotely sensed images, which included red, green, blue, and thermal infrared bands, were used to train an SML algorithm. Our results show that the combination of these four bands can be used to accurately identify foliage pixels within complex field scenes  with an accuracy of 97%, opening up avenues to utilize remotely-sensed thermal infrared observations to characterize plant physiological stress in field conditions.