Combining 3D Shape, Color, and Motion for Robust Anytime Tracking.
David Held, Jesse Levinson, Sebastian Thrun, Silvio Savarese.
Robotics: Science and Systems (RSS), 2014
[Full paper]
[Presentation]
[Poster - pptx]
[Poster - pdf]
Precision Tracking with Sparse 3D and Dense Color 2D Data. Best Vision Paper Finalist.
David Held, Jesse Levinson, Sebastian Thrun
International Conference on Robotics and Automation (ICRA), 2013
[Full paper]
To run a quick test on the code, you will also need to download this file with test data: Test data
If you have any further questions about the method, please email me at davheld -at- cs -dot- stanford -dot- edu.
Using the velocity estimates produced by our tracker, we can accumulate points for a bicyclist as it bikes by:
This is the bicyclist that we are tracking in the above video. Notice that the bicyclist also undergoes rotation as it bikes around the curve.
Here are some other models that are built using the frame-to-frame velocity estimates produced by our tracker. The top row shows the single frame with the most points observed by our sensor. The bottom row shows the accumulated model produced by our tracker.
We can optionally include color in our tracker as well. To do this, we first learn a probability distribution of how much the color of a given point will change over time (from different viewing angles, as shadows change, reflectance, etc). Using the 3D information obtained from our laser and the information obtained from our INS system about our ego motion (i.e. the motion from our camera), we can track a single point as we move past it. We can then observe how the color of this point changes over time, and we can learn the probability of observing any color change over a single frame. This video shows one example point that we are tracking to learn how its color changes as we move past it.
@INPROCEEDINGS{Held-RSS-14, AUTHOR = {David Held AND Jesse Levinson AND Sebastian Thrun AND Silvio Savarese}, TITLE = {Combining 3D Shape, Color, and Motion for Robust Anytime Tracking}, BOOKTITLE = {Proceedings of Robotics: Science and Systems}, YEAR = {2014}, ADDRESS = {Berkeley, USA}, MONTH = {July} }