Hi! I am Desai Xie (谢德赛). I am a 4th-year CS PhD student at Stony Brook University, advised by Prof. Arie Kaufman. I am interning in the Video AI group at Adobe Research in Summer 2024, working on video generation with Yang Zhou and Prof. Feng Liu. I was a research intern in the 3D group at Adobe Research in Summer 2023, working with Xin Sun and Hao Tan. Prior to joining SBU, I spent 3 years at Lehigh University for my undergraduate CS education, where I worked on a protein volume classification project with Prof. Brian Chen.

I am broadly interested in foundation models and sequential decision making. My current research focuses on large-scale pre-training and finetuning of foundation models with synthetic data, e.g. with the data generated by the model itself and the feedback from another model in Carve3D and with procedurally synthesized data in LRM-Zero. I strive to exercise the ideas from the Bitter Lesson in my research projects.

News

I am actively seeking research internship opportunities in late 2024/early 2025, with flexible starting dates and durations. Please reach out if you have any openings!

2024/06/17 Attending CVPR 2024…

2024/06/13 LRM-Zero is released on ArXiv! First time working on large-scale pre-training and data generation, and it was a blast!

2024/05/28 Started my second internship at Adobe Research, this time working on video generation!

2024/02/26 Excited to share that Carve3D is accepted to CVPR 2024!

2023/07/17 My first publication, GAIT, is accepted to ICCV 2023!

2023/06/19 Started my internship at Adobe Research!

Publications

LRM-Zero: Training Large Reconstruction Models with Synthesized Data
Desai Xie, Sai Bi, Zhixin Shu, Kai Zhang, Zexiang Xu, Yi Zhou, Sören Pirk, Arie Kaufman, Xin Sun, Hao Tan
ArXiv 2024
Project Paper Code

Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
Desai Xie, Jiahao Li, Hao Tan, Xin Sun, Zhixin Shu, Yi Zhou, Sai Bi, Sören Pirk, Arie E. Kaufman
Conference on Computer Vision and Pattern Recognition (CVPR), 2024
Project Paper Code

GAIT: Generating Aesthetic Indoor Tours with Deep Reinforcement Learning
Desai Xie, Ping Hu, Xin Sun, Sören Pirk, Jianming Zhang, Radomír Měch, Arie E. Kaufman
International Conference on Computer Vision (ICCV), 2023
Project Paper Code

Misc

A fun fact about my name is that De (德) and Sai (赛) means demoncracy and science in Chinese (Wikipedia).

I love training my “Catificial” Intelligence/CatGPT🐈 agent, Purrari, using a blend of supervised learning (instruction finetuning) and RL (treats as positive reward). She understands many words in both English and Mandarin and has mastered numerous tricks. Currently, she is advancing her communication skills through pet talking buttons. For more cute cat pics and videos, please visit her instagram, lovingly maintained by her mom.