Evaluating your downstream task with Tessera embeddings

tessera-eval turns a ground-truth shapefile into an honest accuracy estimate for Tessera embeddings — cross-validation, learning curves, and spatial splits.

Tessera compresses a year of Sentinel-1 and Sentinel-2 imagery into a single 128-dimensional embedding for every 10 m pixel, which serves as a general-purpose summary of what that patch of ground looked like and how it changed. For an ecologist the practical question is: are these embeddings good enough for my task, and which ML model should I use? The tessera-eval Python package helps to answer that.

Assuming that your ground truth is in a shapefile, the basic idea is that each labelled pixel is matched with a corresponding embedding vector; your labels plus those vectors are an ordinary supervised-learning problem. tessera-eval handles the details, fetching and mosaicking the Tessera tiles, sampling pixels from your polygons, and scoring models. It supports both classification (habitat class, land cover) and regression (canopy height, biomass, percent cover).

What you get back:

• k-fold cross-validation — a robust accuracy estimate (macro-F1, or R²/RMSE/MAE for regression) with the fold-to-fold spread.
• Learning curves — accuracy versus training-set size, so you can see whether collecting more labels would still help.
• Confusion matrices, and predicted-vs-actual scatters for regression.
• Spatial train/test splits — hold out a whole region instead of random pixels, so nearby look-alike pixels can’t inflate the score.
• Spatial-context models — MLPs over a 3×3 or 5×5 neighbourhood, alongside per-pixel k-NN, random forest, XGBoost and MLP.

Here is a minimal run:

import geopandas as gpd
from geotessera import GeoTessera
from tessera_eval import load_embeddings_for_shapefile, run_kfold_cv

gdf = gpd.read_file("habitats.geojson").to_crs(4326)   # polygons with a "habitat" column

# One embedding per 10 m pixel inside your polygons (downloads tiles on first run)
vectors, labels, class_names, stats = load_embeddings_for_shapefile(
    gdf, field="habitat", year=2024, gt_instance=GeoTessera()
)
print(f"{len(labels):,} labelled pixels across {len(class_names)} classes")

# 5-fold cross-validation, random forest on the raw embeddings
for event in run_kfold_cv(vectors, labels, ["rf"], k=5, task="classification"):
    if event["type"] == "aggregate":
        m = event["models"]["rf"]
        print(f"macro-F1: {m['mean_f1']:.3f} ± {m['std_f1']:.3f}")

Replace ["rf"] with ["nn", "rf", "xgboost", "mlp"] to compare models in one pass, or pass task="regression" with a continuous field.

Install with pip install tessera-eval geopandas geotessera. A follow-up post will cover the command-line interface, which runs the same evaluations without writing any Python.

To find out more, here is the link to the git repo.