Projects:Augment marker tracking with visual tracking
Contents
Overview
- Input : Video data, 2D location of birds , 3D trajectory (Labelled and unlabeled) - Output: A. Create 2D trajectories. Object detection on birds. (using simple blob tracking, or using machine learning) B. Match 3D - 2D trajectories to identify identity flips. C. Unlabeled trajectories can be combined with 2D trajectories, to fill gaps in 3D trajectories. (Project 2 output can be useful).
Subprojects: 3.1 Offline data processing; 3.2 Online/quasi-real time solution working from the data stream and video stream (video stream is not directly available in real time for processing, but realtime view is generated in the software, so this could be grabbed)
Contact
Add name of and preferred method how to contact the main PI (i.e. you).
Aims
List the aims of your project, or what you expect anyone taking up the project is supposed to hopefully achieve. The more specific, the better.
Estimated level of difficulty
Generating training data for visual object detection should be a pretty straight-forward standard problem, and is a good way to get into the project and data structures in the framework of a Bachelor/Master project.
An elaborate problem is to find a way to integrate the visual detections into the overall tracking pipeline, since this requires to find a suitable new algorithmic framework. It is closely related to this project.
Provided data
The project uses data from the Vicon system to establish (partially labeled) 3D tracks, as well as input from RGB video cameras. Code for reading the data and calibration, as well as mapping 3D points to 2D images is available (TODO: put on CCU server one git server is up).
Suggested/tested approaches
- to generate training data and build an initial incarnation of the detector:
- find valid segments of 3D trajectories
- use existing code to project 3D tracks into 2D images
- find suitable bounding box and use image crop as a training image
- build database of these and retrain object detector, see Tutorials.
