Hi there! I am Desai Xie (谢德赛). I am a 4th-year CS PhD student at Stony Brook University, advised by Prof. Arie Kaufman. I am interning in the Video AI group at Adobe Research in Summer 2024, working on video generation with Yang Zhou and Prof. Feng Liu. I was a research intern in the 3D group at Adobe Research in Summer 2023, working with Xin Sun and Hao Tan. Prior to joining SBU, I spent 3 years at Lehigh University for my undergraduate CS education, where I worked on a protein volume classification project with Prof. Brian Chen.

I am broadly interested in foundation models and sequential decision making. Currently, I am interested in developing algorithms for video generation models. My recent work, Progressive Autoregressive Video Diffusion Models, is a new algorithm that allows autoregressively generating 60-second videos without quality degradation over time. I have worked on large-scale pre-training and finetuning of foundation models with synthetic data, e.g. with the data generated by the model itself and the feedback from another model in Carve3D and with procedurally synthesized data in LRM-Zero. I strive to exercise the ideas from the Bitter Lesson in my research projects.

News

I am seeking research internship and academia visiting opportunities in late 2024/early 2025, with flexible starting dates and durations. Please reach out if you have any openings!

2024/10/10 Progressive Autoregressive Video Diffusion Models is released on arXiv! Proud to build the first video generation model that can autoregressively generate 60-second videos without quality degradation over time!

2024/09/25 LRM-Zero is accepted at NeurIPS 2024!

2024/06/17 Attended CVPR 2024 in Seattle, WA. I presented Carve3D and was glad to see LRM-Zero being featured in two workshop invited talks (3DFM and SyntaGen) by Hao Tan and Nathan Carr!

2024/06/13 LRM-Zero is released on arXiv! First time working on large-scale pre-training and data generation, and it was a blast!

2024/05/28 Started my second internship at Adobe Research, this time working on video generation!

2024/02/26 Excited to share that Carve3D is accepted at CVPR 2024!

2023/07/17 GAIT is accepted at ICCV 2023!

2023/06/19 Started my internship at Adobe Research!

Publications

Progressive Autoregressive Video Diffusion Models
Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, Yang Zhou
arXiv 2024
Project Paper Code

LRM-Zero: Training Large Reconstruction Models with Synthesized Data
Desai Xie, Sai Bi, Zhixin Shu, Kai Zhang, Zexiang Xu, Yi Zhou, Sören Pirk, Arie Kaufman, Xin Sun, Hao Tan
NeurIPS 2024
Project Paper Code

Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
Desai Xie, Jiahao Li, Hao Tan, Xin Sun, Zhixin Shu, Yi Zhou, Sai Bi, Sören Pirk, Arie E. Kaufman
Conference on Computer Vision and Pattern Recognition (CVPR), 2024
Project Paper Code

GAIT: Generating Aesthetic Indoor Tours with Deep Reinforcement Learning
Desai Xie, Ping Hu, Xin Sun, Sören Pirk, Jianming Zhang, Radomír Měch, Arie E. Kaufman
International Conference on Computer Vision (ICCV), 2023
Project Paper Code

Misc

A fun fact about my name is that De (德) and Sai (赛) means demoncracy and science in Chinese (Wikipedia).

I love training my “Catificial” Intelligence/CatGPT🐈 agent, Purrari, using a blend of supervised learning (instruction finetuning) and RL (treats as positive reward). She understands many words in both English and Mandarin and has mastered numerous tricks. Currently, she is advancing her communication skills through pet talking buttons. For more cute cat pics and videos, please visit her instagram, lovingly maintained by her mom.