Biography

Hi there 👋, I’m Dawei Zhu, a 2nd year Ph.D. student at Peking University. I’m supervised by Prof. Sujian Li. Previously, I obtained my bachelor’s degree from School of EECS, Peking University.

My research focuses on Long Context Modeling. I believe that the ability to effectively process extremely long contexts is a crucial aspect for future language models and multi-modal models, ultimately leading us closer to AGI. Besides, I’m also interested in a broad range of topics, including Agent, Alignment, and the advancement of Multi-modal models.

News

• [Apr. 2024] Our new paper LongEmbed: Extending Embedding Models for Long Context Retrieval is out.

• [Jan. 2024] Our PoSE paper get accepted to ICLR 2024! 🎉🎉🎉

• [Sept. 2023] We introduce Positional Skip-wisE training (PoSE) to decouple traing length and target context length, facilitating efficient context window extension of LLMs.

Education

• Ph.D student, School of Computer Science, Peking University (Sept. 2022 - Now)

• B.Sc in Computer Science and Technology, from School of Electronics Engineering and Computer Science, Peking University (Sept. 2018 - Jun. 2022)

Experiences

• Research Intern at Microsoft Research Asia, Mentored by Liang Wang and Nan Yang (Jun. 2023 - Mar. 2024)

Selected Publications

• LongEmbed: Extending Embedding Models for Long Context Retrieval
  Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li. (Preprint 2024) [paper] [repo]

• PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
  Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li. (ICLR 2024) [paper] [repo]

• CoUDA: Coherence Evaluation via Unified Data Augmentation
  Dawei Zhu*, Wenhao Wu, Yifan Song, Fangwei Zhu, Ziqiang Cao, Sujian Li. (NAACL 2024) [paper] [repo]

• RestGPT: Connecting Large Language Models with Real-World Applications via RESTful APIs
  Yifan Song, Weimin Xiong, Dawei Zhu, Cheng Li, Ke Wang, Ye Tian, Sujian Li. (Preprint 2023) [paper] [repo]

• Large Language Models are not Fair Evaluators
  Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, Zhifang Sui. (Preprint 2023) [paper] [repo]

* denote equal contribution

For full publication list please refer to my google scholar or semantic scholar.

Academic Service

Honors and Awards

• “Stars of Tomorrow” from Microsoft (2024)
• Excellent Graduate of Peking University (2022)