I am interested in learning algorithms for interactive and adaptable embodied AI.
My long term vision is to develop robots that can perform multiple tasks around the home and learn new skills from their users.
My research focuses at the intersection of language, vision and actions to enhance real-time perception, motion control and dialog in robots.
Preprint: Survey on General-Purpose Robots via Foundation Models
We shared the first preprint of our survey on Foundational Models in Robotics.
🦾🤖📚we’ve been exploring the landscape of foundational models in robotics—unveiling insights on current trends and open challenges. A must-read for those interested in the path towards general-purpose robotics. #Robotics#FoundationModels#SurveyPaperhttps://t.co/VziYf3VScn
One (sad?) takeaway for me: when we see planning-based and learning methods compare on even footing, in terms of time invested, we basically never see learning-based methods working better.
HomeRobot is 100% a test of generalization, as object *classes* + envs are totally unseen https://t.co/3nBBudNgns
2 papers and 2 workshop works presented at CoRL'23
I did not attend CoRL this year but check out some of our recent work presented by colleagues at the main conference:
1. HomeRobot: Open-Vocabulary Mobile Manipulation
The future of robot butlers starts with mobile manipulation. We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge! - Full robot stack ✅ - Parallel sim and real evaluation ✅ - No robot required ✅👀https://t.co/mggAbRhrLPpic.twitter.com/Wartsmkyyl
Also, check out some of the work at LangRob and Robot Learning Workshops.
3. PromptBook leverages LLMs for generating robot code! More than examples that were used in Code-as-Policies, we explore Instructions, Chain of Thought Prompting and State Estimation. Led by Montserrat Gonzalez and Andy Zeng at Google DeepMind Robotics. Here is the paper on OpenReview.
4. Open X-Embodiment is a huge robotics data collection effort to enable training of Robotic Foundational Models across multi-embodiments, different tasks, and different lab setups.
RT-X: generalist AI models lead to 50% improvement over RT-1 and 3x improvement over RT-2, our previous best models. 🔥🥳🧵
I am excited to start as a student researcher at Google DeepMind, Mountain View. I will be working with Debidatta Dwibedi on end-to-end video conditioned policy learning for robotics.
Check out the HomeRobot, a large-scale sim-to-real mobile manipulation challenge at @NeurIPSConf 2023! More details about the challenge here. You can submit to EvalAI here. Our paper (accepted at CoRL 2023) shows RL and heuristic policies for sim to real transfer and identifies the challenges in the domain.
(1/5) Every home is different, and every person likes things done in their particular way. Therefore, home robots of the future need to both reason about the sequential nature of day-to-day tasks and generalize to user's preferences.
Selected among 224 young researchers to meet laureates in the mathematics and computer science (postponed to Sep 2021); Participated in Virtual HLF 2020.
I was a Mitacs Globalink Research Intern at Simon Fraser University, Burnaby, Canada. I worked with Prof. Oliver Schulte on bayesian optimization algorithms for machine learning. Find our code here.
March 2017,
Citi Women Leader Award (CWLA) Scholarship
Awarded one year of study scholarship (Top 3 among 1200 candidates selected nationwide).
Vidhi Jain, Maria Attarian, Nikhil J Joshi Ayzaan Wahid, Danny Driess, Quan Vuong, Pannag R Sanketi, Pierre Sermanet, Stefan Welker, Christine Chan, Igor Gilitschenski, Yonatan Bisk, Debidatta Dwibedi.
Preprint 2024.
Montserrat Gonzalez Arenas, Ted Xiao, Sumeet Singh, Vidhi Jain, Allen Z. Ren, Quan Vuong, Jacob Varley, Alexander Herzog, Isabel Leal, Sean Kirmani, Mario Prats, Dorsa Sadigh, Vikas Sindhwani, Kanishka Rao, Jacky Liang, Andy Zeng.
Preprint 2023.