Downstream tasks

• Earth Observation datasets for downstream tasks (spreadsheet)

• Tracking forest restoration in Brazil Through re.green, we have access to about ~300 field plots with repeat LiDAR, ALS, and plot data. Felipe Begliomini.

• Crop segmentation Pastis dataset from France. (Validation or Downstream task?)

• Land classification from natural forest, shrubland and grassland ground truth datasets David Coomes will make enquiries about access to these from a global consortium of field plot owners that we can compare with representations.

• Urban Mapping Urban mapping is interesting as it could help us explore the wild/urban divide where a lot of expansion is happening (see article in Nature, 2023). Anil, Andres, and Ronita are looking into urban classification datasets.

• Counterfactual pixel matching The idea is to find representations for a project pixel and to find counterfactuals that share this representation (ideally over many years). This might help us to match pixels much more quickly. However, we will need to figure out how to additionally match on non-spectral components, such as travel time. Mantle uses a two stage approach for matching and ignore travel time and other socio-economic drivers of e.g., deforestation. They first match on representations, then filter by AGB (all over the past 10 years or so). We may also need to think carefully about how to define closeness in the high dimensional space, taking Mahalanobis distance into account, such as by using PCA.

• Species distribution modelling/Robust Area of Habitat (AoH) classifiers for plant biodiversity The goal is to use representations for studying the dynamics of habitat maps and Key Biodiversity Areas, such as different types of grassland where there are hundreds of thousands of inventory plots around the world. We could continue Emily Morris’s thesis work on Proteus SDM.
  • We have (at least) two possibilities to assess/quantify tree diversity:
    1. by identifying the different tree species present in a given area using, for example, a supervised classification approach,
    2. by interpreting the observed spectral-temporal variability as a proxy of the intra-annual evolution of biochemical and structural properties of the observed pixel, which themselves are indicative for various tree species. As the BT representations capture the spectral-temporal variability of a given pixel (and compress this information into a few latent variables), we indirectly "see" different species (age classes etc.) if we observe differences in the BT signature between two pixels.

• Comparing JRC degradation maps with the representations estimate of degradation We have access to the JRC degradation product at 30m resolution, so we could compare JRC determination of degradation with the representation values, perhaps validating on their field plot data.

• Crop yield analysis We have at least two options for yield predictions:
  1. we regress the BT representations against a suitable set of ‘ground truth’ data - this can be official agricultural statistics/census data (or probably less realistic some in-situ measurements). The outcome is a crop-specific yield number in t/ha (and production number if combined with acreage information). When agricultural statistics are used for yield model calibration, we need a suitable crop mask to extract the representations only from areas covered by the crop of interest - obviously, the BT representations are well suited for this. As crops are usually short-lived, we should ideally adapt the BT window length accordingly (currently 1 calendar year).
  2. we regress the BT representations against a remotely derived yield proxy (here the seasonally accumulated fAPAR) to obtain a field ranking for areas of any size. Instead of generating yield numbers in t/ha as in the first approach (i), here we simply rank all agricultural fields of a given crop type, from low to high. Interestingly, this approach does not require any reference data as the yield proxy is derived from physical-based radiative transfer models (RTM). To get the yield proxy, the RTM is first inverted for a small subset of locations in the ROI. This inversion derives the fraction of absorbed photosynthetically active radiation (fAPAR) and its evolution over the growing season (only cloud-free observations). For each selected location, the fAPAR is afterwards integrated over the entire growing season (e.g., the integral under the fitted curve is calculated). This integrated fAPAR is known to correlate very well with crop growth and yield potential (e.g., similar to Monteith’ light use efficiency). Next, a predictive model is learned between the yield proxy and the corresponding BT representations. The trained model is used to estimate the yield proxy for all locations - this permits to rank the agricultural fields. Mantle has done this to map/estimate leakage (livestock) when converting degraded pastures into agroforestry.

• AGB as a function of the distance to the forest edge Clement was able to see degradation effects several km inside the forest, before everything became stable.

• Estimation fungal biodiversity The idea would be to test if representations capture underground fungal biodiversity or level of carbon sequestration, using data from SPUN.

• Using representations to understand land use change If land use changes, we expect representations to also change. The goal of this project is to exploit the BTFM to detect changes in land use. Open questions include how to integrate ongoing observations effectively, and the change detection threshold. We may also need to embed semantics into representations so that small changes in land use result in small changes in the representation (e.g. using Matryoshka representations). This paper on finetuning BTs for change detection "Barlow twin self-supervised pre-training for remote sensing change detection" is relevant.

• Monitoring mangroves We could potentially monitor mangrove growth through changes in the representations, especially if SAR is incorporated. Tom Worthington or Replanet - Tim Coles would be interested.

• Relate representations to AGB Mantle already does this by mapping representations to the GEDI L4A product. They don't see much difference between the full region and the subregions where they have areas of interest. We need to check if this scales out globally instead of just one Brazilian state.

• Monitoring invasive species Invasive plant species tend to dominate their ecosystem when they show up. So their spectral signature may match up well with the BT ones, in particular as they are usually photosynthetically active (greener) over longer periods of time during a year. Given some labels of known-invasive-species signatures, we could map other hotspots around the world where they are appearing similarly. Alec Christie led an OSINT workshop on this in July 2024 that Anil participated in, and there was a lot of interest in quickly spotting unexpected species occurrence changes due to how rapidly (esp. freshwater) invasive species are moving geographically.

• Spotting landfills in West Africa (Nigeria) from space (Charles Emogor)

• Identifying degraded pasture vs good pasture We would need other sources to get the species of animal involved. This is somehow already covered (see above) under Mantle’s yield mapping approach

• Integrating with COMPASS RICS in India, specifically [ https://anil.recoil.org/notes/compass2024-ric-tripreport/CoRE stack]. Anil will arrange a brainstorming session about BTFM+their downstream tasks. Applications for urban environments for example in poverty mapping, climate-related infrastructure mapping, etc. Maddy has some experience in this area and things this could be useful.