SpeciesNet does not seem to work

[SimonKravis]

In v 6.19, output when using SpeciesNet to recognise animals using either Megadetector 5A or Megadetector 1000 Redwood, output shows only Empty or Blank results.

Example outputs are shown below (with all thresholds set to 0.01)

Part of Megadetector 5A json

Part of Redwood.json

Part of Megadetector 5A and SpeciesNet

Part of Megadetector Redwood and SpeciesNet

[petervanlunteren]

Hi @SimonKravis,

Would you be able to share those images with me so I can do some tests here?

peter@addaxdatascience.com

Cheers,

Peter

[petervanlunteren]

Hi @SimonKravis, I have reproduced it and all I can say is that MegaDetector does recognise the animals, but SpeciesNet does not. SpeciesNet adds the classification “Blank”.

AddaxAI works as it should, I guess this is a model issue.

I think @agentmorris would be interested in the images you shared with me!

[agentmorris]

If there are sample images that I should take a look at, please send them to agentmorris@gmail.com . Thanks.

[petervanlunteren]

Have forwarded the images via email to @agentmorris

[agentmorris]

Thanks for sharing!

A few things are happening here, but first, background on “blank” classifications from SpeciesNet… I have found these to be very reliable indicators of MD false positives, i.e., almost always, a “blank” SpeciesNet classification means the object was a rock, stick, etc. And in fact, that’s what’s happening here, SpeciesNet is working quite well! But it’s being masked by a temporary limitation in AddaxAI that is discussed on other threads.

Specifically, what’s happening here is:

• On the image with the larger, more visible monitor lizard, MegaDetector v5a produces two detections: a detection with 17% confidence on the monitor, and a detection of 44% on a very tail-looking log on the left. 17% isn’t bad, given that MegaDetector v5a is not great at reptiles, but it’s less than 44%.
• If you run SpeciesNet on those detections separately, SpeciesNet calls them “lace monitor” and “blank”, respectively, both excellent classifications!
• However, because the script we originally provided for running SpeciesNet (which is what’s used in AddaxAI) only runs SpeciesNet on the highest-confidence detection in each image, you only ever see the SpeciesNet output on the thing that SpeciesNet (correctly) classified as “blank”.

We prepared a new script for running SpeciesNet that removes this limitation, along with several other improvements; Peter is working on integrating that.

Bonus fun fact: as expected, MDv1000-redwood performs much better on this sequence than MDv5a does, since MDv1000-redwood saw lots of additional large reptiles in training. It has no trouble with the large, obvious monitor (which MDv5a struggles with), and it gets the more subtle skink at far higher confidence than MDv5a, although it’s still low confidence: my human eyes honestly didn’t see it until I read Peter’s email more carefully.

With image augmentation and a larger inference size enabled, MDv1000-redwood does well (not perfectly, but well) on this sequence. But don’t take my word for it; the new SpeciesNet script will also allow you to use other MD models with SpeciesNet in AddaxAI, addressing an issue that you raised on another thread, so when that’s available, you’ll get the correct SpeciesNet behavior and you’ll be able to try MDv1000-redwood.

Keep these “problem images” coming, they are very informative!