About Me

I am an assistant professor at CMU in the Robotics Institute. Prior to my appointment at CMU, I worked as a post-doc at UC Berkeley with Pieter Abbeel on deep reinforcement learning for object manipulation. I completed my Ph.D. in computer science at Stanford working with Sebastian Thrun and Silvio Savarese on perception for self-driving cars. I also have a B.S. and M.S. in mechanical engineering from MIT.

You can also download my CV.

My Group

I have the great fortune to be working with a tremendous group of Ph.D. students:

Brian Okorn (RI, co-advised with Martial Hebert)
Xingyu Lin (RI)
Siddharth Ancha (MLD)
Thomas Weng (RI)

Joining my Group

If you are interested in coming to CMU to join my group as a Ph.D. student, there is no need to email me. Just apply to CMU's Ph.D. program! You should apply to either the Robotics Institute Ph.D. program or the Machine Learning Ph.D. program and mention my name in your research statement. After you get accepted, you should contact me to discuss the possibility of working in my group.


Spring 2018: 16-831: Statistical Techniques in Robotics
Spring 2019: 16-881: Deep Reinforcement Learning for Robotics

Research Interests

My research lies at the intersection of robotics, machine learning, and computer vision.

I am interested in developing methods for robotic perception and control that can allow robots to operate in the messy, cluttered environments of our daily lives. My approach is to design new deep learning / machine learning algorithms to understand environmental changes: how dynamic objects in the environment can move and how to affect the environment to achieve a desired task.

I have applied this idea of learning to understand environmental changes to improve a robot's capabilities in two domains: object manipulation and autonomous driving. I am currently working on learning to control indoor robots for various object manipulation tasks, dealing with questions about multi-task learning, robust learning, simulation to real-world transfer, and safety. Within autonomous driving, I have shown how, by modeling object appearance changes, we can improve a robot's capabilities for every part of the robot perception pipeline: segmentation, tracking, velocity estimation, and object recognition. By teaching robots to understand and affect environmental changes, I hope to open the door to many new robotics applications, such as robots for our homes, assisted living facilities, schools, hospitals, or disaster relief areas.

Reinforcement Learning without Ground-Truth State
Xingyu Lin, Harjatin Singh Baweja, David Held

Iterative Transformer Network for 3D Point Cloud
Wentao Yuan, David Held, Christoph Mertz, Martial Hebert

Combining Deep Learning and Verification for Precise Object Instance Detection
Siddharth Ancha*, Junyu Nan*, David Held
Conference on Robot Learning (CoRL), 2019

Adaptive Auxiliary Task Weighting for Reinforcement Learning
Xingyu Lin*, Harjatin Baweja*, George Kantor, David Held
Neural Information Processing Systems (NeurIPS), 2019

Adaptive Variance for Changing Sparse-Reward Environments
Xingyu Lin, Pengsheng Guo, Carlos Florensa, David Held
International Conference of Robotics and Automation (ICRA), 2019

PCN: Point Completion Network - Best Paper Honorable Mention
Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, Martial Hebert
International Conference on 3D Vision (3DV), 2018

Automatic Goal Generation for Reinforcement Learning Agents
Carlos Florensa*, David Held*, Xinyang Geng*, Pieter Abbeel
International Conference on Machine Learning (ICML), 2018

Enabling Robots to Communicate their Objectives
Sandy Han Huang, David Held, Pieter Abbeel, Anca D. Dragan
Autonomous Robotics (AURO), 2018

Reverse Curriculum Generation for Reinforcement Learning
Carlos Florensa, David Held, Markus Wulfmeier, Pieter Abbeel
Conference on Robot Learning (CoRL), 2017

Policy Transfer via Modularity
Ignasi Clavera*, David Held*, Pieter Abbeel
International Conference on Intelligent Robots and Systems (IROS), 2017

Constrained Policy Optimization
Joshua Achiam, David Held, Aviv Tamar, Pieter Abbeel
International Conference on Machine Learning (ICML), 2017

Enabling Robots to Communicate their Objectives
Sandy H. Huang, David Held, Pieter Abbeel, Anca D. Dragan
Robotics: Science and Systems (RSS), 2017

Probabilistically Safe Policy Transfer
David Held, Zoe McCarthy, Michael Zhang, Fred Shentu, Pieter Abbeel
International Conference on Robotics and Automation (ICRA), 2017

Learning to Track at 100 FPS with Deep Regression Networks
David Held, Sebastian Thrun, Silvio Savarese
European Conference on Computer Vision (ECCV), 2016

A Probabilistic Framework for Real-time 3D Segmentation using Spatial, Temporal, and Semantic Cues
David Held, Devin Guillory, Brice Rebsamen, Sebastian Thrun, Silvio Savarese
Robotics: Science and Systems (RSS), 2016
[Project Page]

Robust Single-View Instance Recognition
David Held, Sebastian Thrun, Silvio Savarese
International Conference on Robotics and Automation (ICRA), 2016
Robust Real-Time Tracking Combining 3D Shape, Color, and Motion
David Held, Jesse Levinson, Sebastian Thrun, Silvio Savarese
International Journal of Robotics Research (IJRR), 2016

Combining 3D Shape, Color, and Motion for Robust Anytime Tracking
Robotics: Science and Systems (RSS), 2014
Precision Tracking with Sparse 3D and Dense Color 2D Data - Best Vision Paper Finalist
International Conference on Robotics and Automation (ICRA), 2013
A Probabilistic Framework for Car Detection in Images using Context and Scale
International Conference on Robotics and Automation (ICRA), 2012
  • Older Work
  • Characterizing Stiffness of Multi-Segment Flexible Arm Movements
    David Held, Yoram Yekutieli, Tamar Flash
    International Conference on Robotics and Automation (ICRA), 2012
    Towards fully autonomous driving: Systems and algorithms
    Jesse Levinson, Jake Askeland, Jan Becker, Jennifer Dolson, David Held, Soeren Kammel, J. Zico Kolter, Dirk Langer, Oliver Pink, Vaughan Pratt, Michael Sokolsky, Ganymed Stanek, David Stavens, Alex Teichman, Moritz Werling, and Sebastian Thrun
    Intelligent Vehicles Symposium (IV), 2011.
    MVWT-II: The Second Generation Caltech Multi-Vehicle Wireless Testbed
    Zhipu Jinh, Stephen Waydo, Elisabeth B. Wildanger, Michael Lammers, Hans Scholze, Peter Foley, David Held, Richard M. Murray
    American Control Conference (ACC), 2004
    Surface waves and spatial localization in vibrotactile displays
    Haptics Symposium, 2010
    Characterization of Tactors Used in Vibrotactile Displays
    Journal of Computing and Information Science in Engineering, 2008

    Elliot Dunlap Smith Hall (EDSH), Room 213