Nordic wildlife detection | Nordic Gradient

The problem

A camera placed in the Nordic countryside should tell which animal passes by: boar, elk, moose, roe deer or fox, or one of the pets, cat and dog. It runs on a Raspberry Pi 3, a cheap computer with low power use, instead of a heavy GPU setup.

The Pi runs the model on its processor, where even the smallest YOLO model needs seconds per picture. That settles the model before any training starts: YOLO26n, the nano version, at 640 pixels. A larger model or a higher resolution is not an option, so every gain has to come from the data.

The approach

A first prototype in one line

Before collecting any data, the simplest thing to try is a pretrained YOLO26n, downloaded and run as it is:

YOLO("yolo26n.pt").predict(frame, conf=0.40)

The real test is 18 animals from my own camera, labeled by hand from about six visits: 9 moose, 6 roe deer and 3 foxes. They are never used for training. COCO, the data it was trained on, has cat, dog, horse, cow and bear, but no moose, deer or fox. It finds 13 of the 18 animals and names none of them correctly: moose become horses, roe deer and foxes become dogs.

Three pictures from my camera with the pretrained model’s predictions: a fox on the lawn called a dog at 83%, a roe deer in snow called a dog at 71%, and a moose called a horse at 91%.

If all I wanted was to catch the moments when an animal passes, this would almost do. To tell a moose from a roe deer, the model has to be fine-tuned on these classes.

The data, and what was wrong with it

The training data combines cats, dogs and birds from COCO with wildlife datasets from Roboflow: trail camera pictures of elk, moose, roe deer and boar, many of them infrared night shots, and ordinary daytime photos of boar and fox. All layers of the model are trained, since trail camera pictures look different from COCO photos. Two problems in the data had to be fixed first.

The camera stripe gives away the answer. Trail cameras stamp a stripe with a logo at the bottom of every picture, and no other class has it. A model can learn “stripe means elk, moose or deer” instead of looking at the animal. The stripe is cropped from every such picture, by an amount found for each one.

Two trail camera pictures, a moose at night and a roe deer in snow, each with the stripe at the bottom marked as the region that is cropped away: 36 pixels of a dark stripe and 72 pixels of a white stripe with a tall logo.

Some animals are too small for a nano model. Run on my COCO validation pictures before any training, the pretrained model finds 2 of 97 bird boxes below 16 pixels, and 23 of 28 at 96 pixels and above. It has already seen all of COCO, so more data would not change that. COCO pictures with any box below 32 pixels are removed, and bird is dropped as a class.

Bar chart of the share of labeled boxes the pretrained model finds, by box size and class. Below 16 pixels it finds 2 of 97 birds. At 96 pixels and above it finds about 80% of birds, cats and dogs.

A good test score, and every animal a dog

The first fine-tuned model looks good on its test set, with a precision of 0.89, a recall of 0.78 and an F1 score of 0.83. On my own camera it finds 14 of the 18 animals, one more than the pretrained model, but still names none of them correctly. It calls every one of them a dog.

The same three pictures with the first fine-tuned model: the fox called a dog at 80%, the roe deer called a dog at 75%, and the moose not found at all.

The reason is the size of the animals. My camera looks out over an open lawn, where a roe deer or a fox covers 1 to 2% of the picture. The wildlife photos show animals at close range. In the training data a wild animal rarely covers less than 2% of the picture, while small dogs are common, so the model learned “a small animal in a wide scene is a dog”. The test set comes from the same sources as the training data, so it could not show this.

Bar chart of training boxes that cover less than 2% of the image, per class: 22% for dog, 12% for cat, and between 3 and 11% for the wild animals.

Segmentation pasting into my own scenes

The remedy is to paste animals from the training set, small, into empty pictures from my camera. A segmentation model, MobileSAM, turns the box around an animal into its outline, so no rectangle of forest comes along. The pasted size follows the place and the species: further up the lawn is further away and smaller, and at the same place a moose is about twice as tall as a roe deer. Some empty pictures are added unchanged, as scenes without any animal. The figure shows the same steps on public training pictures.

Segmentation pasting in two rows. On the left, MobileSAM outlines a roe deer in grass and a fox in snow. On the right, each animal is pasted into a trail camera picture at a size that matches the animal already there.

Two more datasets fill the gaps the camera test showed: moose in daylight, where the other moose are mostly gray infrared shots, and roe deer, fox and boar from Italian trail cameras.

Results

On the test set the final model gains a little on the first one. Each runs at the confidence threshold that gives its best F1 score on the validation set, 0.40 for the final model:

Model	Precision	Recall	F1 score
First model	0.89	0.78	0.83
Final model	0.90	0.84	0.87

On my own camera the difference is large. The final model names 16 of the 18 animals correctly: all 3 foxes, all 6 roe deer and 7 of 9 moose. One moose is called a roe deer, and one, of which only the legs show at the edge of the picture, is missed.

Four pictures from my camera with the final model: the fox at 86%, the roe deer at 64% and a moose at 80% are named correctly. In the fourth, a moose is named roe deer at 69%.

Most of the time my camera sees no animal at all, and a model that sees animals there is of little use. So the test also has 108 pictures from my camera without any animal, never used for training. All three models run at 640 pixels and a threshold of 0.40:

Model	Animals found	Animals named correctly	Empty scenes left empty
Pretrained YOLO26n	13 of 18	0 of 18	108 of 108
First model	14 of 18	0 of 18	105 of 108
Final model	17 of 18	16 of 18	108 of 108

The final model finds and names the animals without seeing animals where there are none. On the test set, the wild animals are almost never mistaken for each other.

Predictions on six test images: boar, moose, elk, roe deer, fox and cat, found at 69 to 87% confidence with boxes close to the labeled ones. One of three boar is missed.

Where it fails

Cats and dogs are the weak spot on the test set. The model mixes them up, with 10 cats called dogs and 17 dogs called cats, which matters little here, since both are pets. More often it misses them entirely: 24 of 116 cats and 101 of 317 dogs are not found. Of the wild animals, 27 of 311 moose and 16 of 132 boar are missed, and elk, roe deer and fox only rarely.

Confusion matrix on the test set. The wild animals sit almost only on the diagonal. 101 dogs and 24 cats fall under background, meaning they were not found.

Hard pictures show where the limits are. The model keeps a roe deer whole on both sides of a tree, and finds an elk at night at 48%, low but above the threshold. In a blurry low-resolution frame it finds the fox but also a boar across almost the whole picture, and two roe deer that overlap in the vegetation become one.

Four hard test images: a roe deer behind a tree, an elk at night, a fox in a blurry frame with a false boar, and two overlapping roe deer detected as one.

And there are known limits:

On the Raspberry Pi 3 the model takes about 2 seconds per picture, so an animal that passes quickly is seen in few pictures or none.
The trail camera pictures in training and test come from the same camera locations. The scores say how the model does at places it has seen, not at a new one.
The test on my own camera rests on 18 pictures from a few visits, so a difference of one or two pictures says little.
The 108 empty scenes come from the same weeks, the same season, as the empty pictures used for training, so they are easier than a new season would be.
Animals outside the seven classes can still be detected: the final model calls one of two hares on my camera a dog.

Status

Fine-tuned and tested on my own camera. Next, more animals from the camera are collected and labeled, as an honest test set and as training data, and the model is tested on empty scenes from a new season. After that, a home automation integration is planned, to send a notification to the phone when an animal passes.

The pictures come from COCO and the Roboflow datasets Wild-deer, Wildlife and Animal Detection, and from moose-detection-8-cl-v2 by test-q5dxc and DS by l-ras, both CC BY 4.0. The pictures of the fox, the roe deer and the moose that appear in all three figures from my own camera are cropped, levelled and mirrored, and a windmill is retouched out.

Code: github.com/johndah/object-detection. The notebook also runs on Kaggle.