Improved approach to label-free identification using segmented tracklets for object behavior analysis

Abstract

In this work we present an improved approach to label-free identification using segmented tracklets for object behavior analysis - Mask-TAUDL. This approach takes segmentation into account and combines a two-stream Mask R-CNN detector/segmenter (twinResNet18) with Tracklet Association Unsupervised Deep Learning (TAUDL). High-quality instance masks and merged features improve image clarity and appearance consistency, while TAUDL simultaneously trains discriminative embeddings and associations across sessions without identity labels. The proposed approach improves identification robustness under occlusions, pose changes, and lighting changes, reducing identity switching and fragmentation during long-term observations. The integration of pure masked fragments, motion features, and temporal enhancements in TAUDL enables scalable, annotation-free tracking. Reliable animal reidentification in long-term behavioral studies remains challenging due to occlusions, lighting variations, and visual similarity between individuals. Most existing methods rely on pre-labeled datasets, limiting scalability in laboratory environments. The proposed approach enhances robustness under occlusions, pose changes, and illumination drift, reducing identity switches and fragmentation during long-term observations. The approach is intended for studying animal behavior, particularly mice and fish, that are monitored in closed or controlled laboratory environments, where stable long-term tracking is required for behavioral analysis.