A vision-based temporal cross-view synchronization framework for sports user generated videos comparing motion and semantic representations

Abstract

This dissertation introduces Track-UGV-Sync, a search-efficient and object-centric framework for temporal alignment based exclusively on visual features embeddings. The framework combines (i) multi-object tracking to extract significant object trajectories, (ii) visual correspondence using ReID or self-supervised representations, and (iii) robust temporal offset estimation through similarity search and RANSAC (RANdom SAmple Consensus). Its evaluation comprises 531 video pairs from eight sports and three viewpoint conditions. Within the object-centric pipeline, replacing identity-oriented OSNet-AIN embeddings with DINOv2 representations reduced the Mean Absolute Error from 65.33 to 54.80 frames and increased Acc@5 from 8.47% to 13.75%. A complementary comparison with whole-frame representations showed that DINOv2 combined with Dynamic Time Warping achieved the best aggregate visual result, with an MAE of 36.39 frames and Acc@5 of 49.72%. Conversely, object-centric DINOv2 with RANSAC achieved the best performance for same-view pairs, reaching an MAE of 14.78 frames and Acc@5 of 60.87%. These findings indicate that global scene representations are more robust under large viewpoint differences, whereas tracklet representations provide more discriminative temporal evidence when object appearance remains comparable. By restricting the search to frames belonging to significant trajectories, Track-UGV-Sync reduced the number of candidate offset hypotheses from hundreds of thousands to just hundreds. This yields a 93.4× wall-clock speedup.