It’s September, 2022, and we’ve released a new computer vision model for iNaturalist. This follows updates in August and April 2022. The iNaturalist website, iNaturalist mobile apps, and API are all now using this new model. Here’s what’s new and different with this change:

• It includes 65,000 taxa (up from 60,000)
• It is the second model to be trained in our new, faster approach

Taxa differences to previous model

There are 5,811 taxa in the new model (v1.2) that weren’t in the old model (v1.1).

4,717 of those represent newly added choices. For example, of the 3 species of Carpillus, the old model only included Spotted Reef Crab and Convex Crab whereas the new model also includes Batwing Coral Crab.

907 of those taxa represent more refined replacements. For example, the old model included the Crestless Curassow genus Mitu which contains 4 species of birds. None of these species had enough photographs to be included in the model. The new model includes the species Razor-billed Curassow as a more refined replacement for Mitu. Because genus Mitu was replaced by Razor-billed Curassow, the number of choices was not increased by this refinement.

Lastly, 187 of thetaxa in the new model but not in the old model result from taxon changes. For example, in the old model Wedge-rumped Storm-Petrel was represented by the taxon Oceanodroma tethys, but due to a taxon change the new model has replaced that taxon with Hydrobates tethys. This is the same species but in a different genus so again the number of choices was not increased by this refinement.

There were also 1,165 species in the old model which are not in the new model. 31 of these were lost because of a decrease in the amount of data. For example, the old model included 2 species of genus Aplysilla, Encrusting Rose Sponge (Aplysilla rosea) and Aplysilla glacialis. However, due to new identifications added by the community, many of the observations that were identified as Encrusting Rose Sponge now represent other taxa. As a result, the new model no longer includes this taxon as a node.

As described above, there are also taxa in the older model not in the newer model because they were refined (e.g. genus Mitu) or because they were the inputs of taxon changes (e.g. Oceanodroma tethys)

The charts below summarize these taxa. We can use these categories to filter out just this set of 4,717 new taxa added to the new model that aren’t the result of refinements or taxon changes.

By category, most of these 4,717 new taxa were insects and plants.

Here are species level examples of new species added for each category:

• Other Animalia
• Mollusca
• Plantae (sample of 200)
• Amphibia
• Protozoa
• Mammalia
• Actinopterygii
• Insecta (sample of 200)
• Chromista
• Arachnida
• Aves
• Fungi
• Reptilia

Click on the links to see these taxa in the Explore page to see these samples rendered as species lists.

You can find an entire list of all the species added to the new model here.

Remember, to see if a particular species is included in the currently live computer vision model, you can look at the “About” section of its taxon page.

This is our new vision model release tempo

Our previous goal for releasing models was twice a year, and we struggled to even meet that. However, with the new transfer learning approach that vastly speeds up training, we now plan to release a model every month, with the caveat that our schedule could grow longer as the number of photos continues to grow. This means that there will be much less taxonomic drift between the taxonomy that the model knows about and the taxonomy at the time the model is showing suggestions to a user.

We will still be training a full model once or twice a year, which we’ll then do transfer learning from in order to make release models. Extra hardware provided by NVIDIA and donations from the iNat community have made it possible to have a training strategy that combines both full model training and transfer learning.

Future work

First, we are still working on new approaches to improve suggestions by combining visual similarity and geographic nearness. We still can’t share anything concrete, but we are getting closer.

Second, we’re still working to compress these newer models for on-device use. The in-camera suggestions in Seek continue to use the older model from March 2020.

We couldn't do it without you

Thank you to everyone in the iNaturalist community who makes this work possible! Sometimes the computer vision suggestions feel like magic, but it’s truly not possible without people. None of this would work without the millions of people who have shared their observations and the knowledgeable experts who have added identifications.

In addition to adding observations and identifications, here are other ways you can help:

• Share your Machine Learning knowledge: iNaturalist’s computer vision features wouldn’t be possible without learning from many colleagues in the machine learning community. If you have machine learning expertise, these are two great ways to help:
• Participate in the annual iNaturalist challenges: Our collaborators Grant Van Horn and Oisin Mac Aodha continue to run machine learning challenges with iNaturalist data as part of the annual Computer Vision and Pattern Recognition conference. By participating you can help us all learn new techniques for improving these models.
• Start building your own model with the iNaturalist data now: If you can’t wait for the next CVPR conference, thanks to the Amazon Open Data Program you can start downloading iNaturalist data to train your own models now. Please share with us what you’ve learned by contributing to iNaturalist on Github.
• Donate to iNaturalist: For the rest of us, you can help by donating! Your donations help offset the substantial staff and infrastructure costs associated with training, evaluating, and deploying model updates. Thank you for your support!

Our Observation of the Week is this trio of Predatory Fungus Gnats (in the genus Heteropterna), seen in Singapore by @airgel!

In early August, Sam (@airgel) and some nature guides went for a night walk on a dirt road named Track 15. “It's quite a popular spot for night walks here in Singapore,” says Sam.

I spotted these flies (presumably) sleeping, hanging from this strand of silk - like tiny line dancers! I called my friends over to take a look - but after just three shots they had disappeared. I feel kinda bad for waking them up, but if the picture can be used to educate and contribute to research, I suppose it's worth it. I was also fortunate enough to have the shots in focus - being able to make out wing venation is really important for identifying a whole host of insects.

When I uploaded the photo as an observation, I had no idea what type of fly I had seen. Next thing I knew, it was Observation of the Day and the identifications started coming in! I'm humbled and grateful for everyone's help with identifying these gnats.

As you’d suspect from their common name, predatory fungus gnats (Family Keroplatidae) can feed on both fungi and other animals during their larval stage. According to BugGuide

they spin hygroscopic webs to collect spores or small invertebrate prey. Predaceous species kill their prey with an acid fluid (mostly oxalic acid) secreted by labial glands and deposited in the droplets of their web; mycophagous larvae also have acid webs and occasionally feed on pupae of their own species or on dead insects. 

iNat user @treegrow suspects that the trio Sam photographed, which appear to be males, are hanging on spider silk and are likely looking for receptive females.

In addition to the gnats, Sam also saw his first Cyrtarachne bird dropping mimic spider and Bipalium hammerhead flatworms that night.

While Sam (above) has always been into nature (“there are pictures of me as a toddler staring at ants on the ground”), he credits his first macro lens as the impetus for his “deep dive” into the natural world. 

As I started looking for subjects to photograph, I learnt more and more and fell ever deeper in love with nature. My interests broadened when I got involved with a group of nature guides in my university - I started learning more and getting more into the conservation and outreach side of things. But my focus will always be my first love - inverts! I study aerospace engineering, but hope to contribute to research efforts some day - I still have a lot to learn!

He says that for about two years his photos mostly lived on his hard drive but a persistent friend finally persuaded him to start posting his photos to iNat this year.

It's been awesome! With iNaturalist, I've been able to learn so much more about the life I photograph. I especially love it when the experts come in and have discussions in the comments, on top of providing identifications. I think that's where the greatest potential for learning lies for me!

I think iNaturalist has also shaped the direction of my photography. On top of thinking about things like colour, composition and ethics, I've started thinking - how can I get the most scientifically useful and accurate shot? For instance, I've come to prefer shooting the dorsal view of jumping spiders instead of the classic anterior face-on view, because I find dorsal views more useful in identification. And the more identifiable something is, the more useful the observation.

I also think iNaturalist has broadened my focus. I've recently started to take more notice of plants, and although I have next to no relevant knowledge, the wonderful local identifiers have made it really easy to learn more.

(Photo of Sam by @phoebezhouhuixin.)

- You can follow Sam on Instagram here.

- Perhaps the most famous member of Family Keroplatidae is the New Zealand Glowworm. Check out PBS Deep Look’s video about them.

Our Observation of the Day is this Asterella drummondii liverwort, seen in Australia by @knicolson!

Ah, liverworts. They are small non-vascular terrestrial plants, but I think in general people are less familiar with them than they are with mosses, another large group of non-vascular plants. However, Australian botanist Kym Nicolson is a fan. “Small is beautiful,” he says. “I have had a fascination with small ephemeral plants for many years. The small often non-colourful, non-spectacular plants that you can’t see unless you get down on your hands and knees, with your face close to the ground. The things you don’t see when walking. You have to stop and look.”

Earlier this month, Kym visited Belair National Park, outside of Adelaide, to test a new camera.  

As it had been a wet spring, there were patches of moss and small ephemerals.  I was photographing some very small Centrolepis plants and then realised alongside was a miniature forest of fruiting bodies of the liverwort Asterella drummondii, which I then also proceeded to photograph. I have only seen these fruiting bodies a couple of times before and only once in this National Park. Liverworts are often overlooked due to their size.

It's a challenge photographing such small objects and I like to place the camera on the ground on a small beanbag to get a lateral view rather than photograph from above. The photos often show detail you can't see by just looking.

One of the advantages of getting close to the ground is you see a whole world of biodiversity you did not know existed. Tiny spiders, mites, ants and an array of other insects. I recently saw a peacock spider and am now trying to resist the temptation to get interested in the Salticidae.

As Kym noted above, his photograph shows the fruiting body Asterella drummondii. When looked at from above, you’ll see they generally have a single long thallus from which the fruiting body emerges.

Kym (above) says he’s had a lifelong interest in plants, which motivated him to earn a PhD in Botany at the University of Adelaide. 

For years I photographed and identified plants for my own personal interest, just as a hobby. The advent of iNaturalist has provided a purpose and a repository for my observations. I see myself as a contributor to iNaturalist rather than an end user of the observational data. A citizen scientist contributing observational data on our valuable biodiversity.

I have a particular interest in grasses and the Australian Chenopodiaceae (now placed in the Amarathaceae) and take every opportunity to try and photograph species I haven't previously seen and add them to iNaturalist.

- The iNat community has posted over 100,000 liverwort observations, check them out!

- Gardening Australia’s got this nice video about non-vascular plants.