Vibhav Vineet

My research interests are in computer vision, machine learning, and human-AI interactions. My ongoing research is focused on the development of models that utilize multi-modal data to enhance AI systems' ability to perceive and reason about the real-world environments of human users. This advancement will ultimately help AI systems to seamlessly interact and collaborate with humans, accomplishing tasks within the real world efficiently.
Topics of interest. 1) Multi-modal robustness analysis and reasoning. 2) Human-AI multi-modal interactions.

If you are interested in research collaborations or doing research internship at MSR Redmond, please contact me.

Recent and Selected Publications

• DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
  Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet .
  Neural Information Processing System (NeurIPS), 2023.
  

  [Paper]

• Revealing the unseen: Benchmarking video action recognition under occlusion
  Shresth Grover, Vibhav Vineet, Yogesh Singh Rawat .
  Neural Information Processing System Dataset and Benchmark track (NeurIPS), 2023.
  

  [Paper]

• On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes
  Rajat Modi, Vibhav Vineet, Yogesh Singh Rawat .
  Neural Information Processing System Dataset and Benchmark track (NeurIPS), 2023.
  

  [Paper]

• Efficiently Robustify Pre-Trained Models
  Nishant Jain, Harkirat Behl, Yogesh Rawat, Vibhav Vineet .
  International Conference on Computer Vision (ICCV), 2023.
  

  [Paper]

• YCB Digital Twins for Sim2Real Analysis
  Sruthi Sudhakar, Jon Hanzelka, Josh Bobillot, Tanmay Randhavane, Pedro Urbina, Neel Joshi, Vibhav Vineet .
  International Conference on Computer Vision (ICCV), 2023.
  

  [Paper]

• Controllable Text-to-Image Generation with GPT-4
  Tianjun Zhang, Yi Zhang, Vibhav Vineet, Neel Joshi, Xin Wang .
  arXiv, 2023.
  

  [Paper]

• Robustness Analysis on Foundational Segmentation Models
  Madeline Chantry Schiappa, Sachidanand VS, Yunhao Ge, Ondrej Miksik, Yogesh S Rawat, Vibhav Vineet .
  arXiv, 2023.
  

  [Paper]

• A Large-Scale Robustness Analysis of Video Action Recognition Models
  Madeline Chantry Schiappa, Naman Biyani, Prudvi Kamtam, Shruti Vyas, Hamid Palangi, Vibhav Vineet, Yogesh S Rawat .
  IConference on Computer Vision and Pattern Recognition (CVPR), 2023.
  

  [Paper]

• Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
  Madeline Chantry, Shruti Vyas, Hamid Palangi, Yogesh Rawat, Vibhav Vineet .
  Advances in Neural Information Processing Systems (NeurIPS dataset track), 2022.
  

  [Paper]

• 3db: A framework for debugging computer vision models
  Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, Ashish Kapoor, Aleksander Madry .
  Advances in Neural Information Processing Systems (NeurIPS), 2022.
  

  [Paper]

• Neural-Sim: Learning to Generate Training Data with NeRF
  Yunhao Ge, Harkirat Behl, Jiashu Xu, Suriya Gunasekar, Neel Joshi, Yale Song, Xin Wang, Laurent Itti, Vibhav Vineet .
  European Conference on Computer Vision (ECCV), 2022.
  

  [Paper]

• MTFormer: Multi-Task Learning via Transformer and Cross-Task Reasoning
  Xiaogang Xu, Hengshuang Zhao, Vibhav Vineet, Ser-Nam Lim, Antonio Torralba .
  European Conference on Computer Vision (ECCV), 2022.
  

  [Paper]

• Missingness bias in model debugging
  Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, Aleksander Madry .
  International Conference on Learning and Representation (ICLR), 2022.
  

  [Paper]

• Benchmarking spatial relationships in text-to-image generation
  Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang .
  arXiv, 2023.
  

  [Paper]

• Dall-e for detection: Language-driven context image synthesis for object detection
  Yunhao Ge, Jiashu Xu, Brian Nlong Zhao, Laurent Itti, Vibhav Vineet .
  arXiv, 2022.
  

  [Paper]

• Learning Articulated Rigid Body Dynamics Simulations From Video
  Eric Heiden, Ziang Liu, Vibhav Vineet, Erwin Coumans, Gaurav Sukhatme .
  ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality, 2022.
  

  [Paper]

• AutoSimulate:(Quickly) Learning Synthetic Data Generation
  Harkirat Singh Behl, Atılım Güneș Baydin, Ran Gal, Philip HS Torr, Vibhav Vineet.
  European Conference on Computer Vision (ECCV), 2020.
  

  [Paper]

• Playing for data: Ground truth from computer games
  Stephan R Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun.
  European conference on computer vision (ECCV), 2016.
  

  [Paper]

• Feature space optimization for semantic video segmentation
  Abhijit Kundu, Vibhav Vineet, Vladlen Koltun.
  Conference on computer vision and pattern recognition (CVPR), 2016.
  

  [Paper]

• Semanticpaint: Interactive 3d labeling and learning at your fingertips
  Julien Valentin, Vibhav Vineet, Ming-Ming Cheng, David Kim, Jamie Shotton, Pushmeet Kohli, Matthias Nießner, Antonio Criminisi, Shahram Izadi, Philip Torr.
  ACM Transactions on Graphics (TOG), 2015.
  

  [Paper]

• The semantic paintbrush: Interactive 3d mapping and recognition in large outdoor spaces
  Ondrej Miksik, Vibhav Vineet, Morten Lidegaard, Ram Prasaath, Matthias Nießner, Stuart Golodetz, Stephen L Hicks, Patrick Pérez, Shahram Izadi, Philip HS Torr.
  ACM Conference on Human Factors in Computing Systems (CHI), 2015.
  

  [Paper]