Projects:Augment marker tracking with visual tracking

From Collective Computational Unit
Revision as of 08:38, 21 May 2019 by Admin (talk | contribs) (→‎Suggested/tested approaches)
Jump to navigation Jump to search

Contents

Overview

- Input : Video data, 2D location of birds , 3D trajectory (Labelled and unlabeled) - Output: A. Create 2D trajectories. Object detection on birds. (using simple blob tracking, or using machine learning) B. Match 3D - 2D trajectories to identify identity flips. C. Unlabeled trajectories can be combined with 2D trajectories, to fill gaps in 3D trajectories. (Project 2 output can be useful).

Subprojects: 3.1 Offline data processing; 3.2 Online/quasi-real time solution working from the data stream and video stream (video stream is not directly available in real time for processing, but realtime view is generated in the software, so this could be grabbed)

Contact

Add name of and preferred method how to contact the main PI (i.e. you).


Aims

List the aims of your project, or what you expect anyone taking up the project is supposed to hopefully achieve. The more specific, the better.


Estimated level of difficulty

Generating training data for visual object detection should be a pretty straight-forward standard problem, and is a good way to get into the project and data structures in the framework of a Bachelor/Master project.

An elaborate problem is to find a way to integrate the visual detections into the overall tracking pipeline, since this requires to find a suitable new algorithmic framework. It is closely related to this project.

Provided data

The project uses data from the Vicon system to establish (partially labeled) 3D tracks, as well as input from RGB video cameras. Code for reading the data and calibration, as well as mapping 3D points to 2D images is available (TODO: put on CCU server one git server is up).

Suggested/tested approaches

  • to generate training data and build an initial incarnation of the detector:
    • find valid segments of 3D trajectories
    • use existing code to project 3D tracks into 2D images
    • find suitable bounding box and use image crop as a training image
    • build database of these and retrain object detector, see Tutorials.
  • talk to people working on tracking for ideas on how to integrate visual and marker detections.
Retrieved from "https://ccu.uni-konstanz.de:443/w/index.php?title=Projects:Augment_marker_tracking_with_visual_tracking&oldid=308"