Xilun Zhang

I am a Master Student at Carnegie Mellon University SafeAI Lab supervised by Ding Zhao. I work closely with Peide Huang and Mengdi Xu. Previously, I was a Research Assistant at Multi-Agent Robotic System Lab with Prof.Mo Chen. I completed my bachelor's degree with distinction from Simon Fraser University School of Mechatronic Systems Engineering, and completed my Capstone Project on self-driving cars in Autonomous & Intelligent Systems Lab with Prof. Ahmad Rad.

I am passionate about intelligent decision-making algorithms with applications in Robotics. I am particulary interested in dynamic adaptations, skill learning, Robot Foundation Models.
I am activetly looking for phd opportunities starting in Fall 2025.

Email / CV / Google Scholar / Linkedin / Twitter

News

2024/06 - I am joining Esperanto Technologies as a Machine Learning Engineer Intern, working on open-source VLMs in Robotics and Robot Foundation Models.
2023/08 - Our paper about using LLMs+Robotics+Tool Use is online now! Also check the twitter post for interesting demos and join the discussion!
2023/08 - Our paper about closing the sim-to-real gap thourgh causal discovery got accepted by CoRL 2023 for a poster presentation, congratulations to Peide!
2023/08 - Our paper about continual vision-based RL got accepted by CoRL 2023 for an oral presentation, congratulations to Shiqi and Mengdi!

Publications

	Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications Xilun Zhang, Shiqi Liu, Peide Huang, William Jongwon Han, Yiqi Lyu, Mengdi Xu, Ding Zhao arxiv, submitted to RA-L. indicates equal contribution.* [paper] [webpage]
	Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances Hanyang Hu, Xilun Zhang, Xubo Lyu, Mo Chen arxiv, submitted to ICRA 2025. [paper]
	Creative Robot Tool Use with Large Language Models Mengdi Xu, Peide Huang, Wenhao Yu, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao arxiv, under review * indicates equal contribution.* Abridged in CoRL 2023 Workshop on Language and Robot Learning. [paper] [webpage]
	Continual Vision-based Reinforcement Learning with Group Symmetries Shiqi Liu, Mengdi Xu, Peide Huang, Xilun Zhang, Yongkang Liu, Kentaro Oguchi, Ding Zhao Conference on Robot Learning (CoRL), 2023. (oral, 6.6%)* * indicates equal contribution.* Abridged in RSS 2023 Workshop on Symmetries in Robot Learning. [paper] [webpage]
	What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discovery Peide Huang, Xilun Zhang, Ziang Cao, Shiqi Liu, Mengdi Xu, Wenhao Ding, Jonathan Francis, Bingqing Chen, Ding Zhao Conference on Robot Learning (CoRL), 2023. * indicates equal contribution.* Abridged in IROS 2023 Workshop on Causality for Robotics: Answering the Question of Why. [paper] [webpage]

Teaching

Course Assistant of 24-767 Modern Control: Theory and Design, Carnegie Mellon University, Fall 2023
Course Assistant of 24-784 Trustworthy AI Autonomy, Carnegie Mellon University, Spring 2024

Service

Conference Reviewer: ICRA 2025
Journal Reviewer: RA-L

Projects

	Generalizable BeamWalking for Legged Robots with Reinforcement Learning Xilun Zhang, Changyi Lin, Zongyuan Wu, Jian Chen 16-831: Intro to Robot Learning(by Deepak Pathak) [paper]
	Enhancing Robustness of Reinforcement Learning using Hamilton-Jacobi Reachability Xilun Zhang 16-886: Models & Algorithms for Interactive Robotics(by Andrea Bajcsy)

Website Template