I have presented the current research activities in our Image and Video Understanding Laboratory at KAUST. The applications range from video understanding to autonomous navigation, with solid fundamentals in deep learning theory. First, I introduced the IVUL research on Video Understanding, including pioneer work on activity proposals and localization, video retrieval and visual object tracking. Then, I presented our works on Vision for Automation, focusing on 3D computer vision and imitation learning applied on self navigation, underlining the importance of simulation and transfer learning. Last, I presented the on-going research on Deep Learning Fundamentals, including large scale optimization, analytical expression for neural networks and robustness to adversarial attack.