“Show me your neighbors, and I’ll tell who you are”
“Show me your neighbors, and I’ll tell who you are”
By Sergio Marconi
How many times Hollywood showed scientists in their hoodies tracking a person of interest, hacking security cameras around the world? Sitting for hours on huge machines, weird geniuses whose typing in a terminal is far from reproducible.
Actually, nowadays, 40 lines of code and a good tutorial (Chollet, 2017), and you can distinguish a person from any other with good reliability in a blink of an eye. For example, the use of convolutional neural networks (convnets) became more and more popular in Visual Recognition, proving to be one of the best classification algorithms, so far. As example, a common task is to distinguish different animals pictured in a photo. Well, believe me or not, you can easily build a model with 95% of accuracy in less
Figure 1 Extract of the 4.5 million geolocated predicted Crowns the NEON site OSBS (FL). Tones of blue represent Nitrogen concentration per individual crown.
than a week.
“Awesome, you’d say”; “why not use them to infer species biodiversity, by discerning species label from any tree in a set of pictures?”
“Because nature is awesomely more complicated, and worthy all the fun trying!”
Differently from classical datasets, classifying trees species from remote sensing imagery is complicated by:
- Being a terribly unbalanced dataset. In fact, for how smart it is, your computer will always try to give the “optimal answer”, which, probabilistically speaking, may be “stick with the same most common guess!”
- Patterns impossible to recognize from human eyes: in short, differently from classical situations, where we asked computers to be as smart as human beings, we are now demanding them to be way smarter;
- Pictures usually contain more than a single object, which could confuse our poor synthetic brain!
- Collecting data in ecology is expensive, way too expensive than other sources. 20000 pictures are considered a “limited and small data set”.
Figure 2 Classifiers are meant to optimize the accuracy rate of guessing. If the sample is strongly unbalanced, the machine may find more useful to always guess Pine instead of learning how to guess Elephant.
“In short: what point are you trying to make here? Is it possible or not?”
Fortunately, we have ideas and hints on how to deal with these issues. From (1) developing a balanced weighted method to penalize misclassification of rare species, to (2) reduce data complexity (dimensionality), to (3) predict and filter which areas in an image are/not area of interest, and (4) automatically extract images of trees localized in other studies, and “create” new images by deforming available ones (data augmentation).
Will these hints be sufficient? Who knows, cross finger for now. But as is, we are able to accurately label around 80% of the trees in Ordway Swisher Biological Station, infer their crown shape and chemical structure for between 50 and 80% of their variability. Not as good as labelling “Cats or Dogs”, but definitively more fun and meaningful to quantify functional diversity!
References:
Chollet, F. (2017). Deep Learning with Python. Manning Publications.
