About Me
Hi, I am Shenggui Li, you can call me Frank Lee as well. I am an incoming PhD student of the 📍S-Lab in Nanyang Technological University @ Singapore, advised by Prof. Tianwei Zhang and Prof. Ivor Tsang. Prior to that, I obtained my 🤓 B.Eng in Computer Science in Nanyang Technological University in 2021 advised by Prof. Chen Change Loy. I also worked with Prof. Yang You from 2021 to 2024 as a research asssistant in the HPC-AI Lab and a founding member of HPC-AI Technology.
My interest lies at the intersection of 🤖 machine learning and ⚡️ high performance computing. I first proposed the concept of sequence parallelism in 2021, which has attracted follow-ups and is widely adopted for large-scale model training. I am the author of Ring Self-Attention and Colossal-AI. During my undergraduate study, I had the privilege to spend a wonderful year on the NTU HPC Team and we won the overall 🥈 2nd place in the student cluster competition of ISC21.
I am involved in several interesting projects, feel free to email me for collaboration.
- 🎥 Open-Sora: Core contributor
- ⚡️ SwiftInfer: Core contributor
- ⚡️ ColossalAI: Top 1 contributor, little involvement now
News
- [2024-04] 🎉 awarded the prestigious A*STAR Computing & Information Science Scholarship, which will support my PhD studies at NTU.
- [2024-03] 🎥 released the Open-Sora project V1 , congratulations to Zangwei Zheng, Xiangyu Peng, Hongxin Liu and other collaborators and contributors!
- [2024-03] 🎉 admitted into the 2024 Fall PhD programme of NTU under the supervision of Prof Tianwei Zhang.
- [2024-01] 🚀 released SwiftInfer , a TensorRT implementation of StreamingLLM.
Selected Publications
- Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
Shenggui Li, Hongxin Liu, Zhengda Bian, Jiarui Fang, Haichen Huang, Yuliang Liu, Boxiang Wang, Yang You,
Proceedings of the 2023 International Conference for Parallel Processing (ICPP 2023)
- Sequence Parallelism: Long Sequence Training from System Perspective
Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, Yang You,
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)
- Parallel Training of Pre-Trained Models via Chunk-Based Dynamic Memory Management
Jiarui Fang, Zilin Zhu, Shenggui Li, Hui Su, Yang Yu, Jie Zhou, Yang You,
IEEE Transactions on Parallel and Distributed Systems (TPDS 2023)
- Online evolutionary batch size orchestration for scheduling deep learning workloads in GPU clusters
Zhengda Bian, Shenggui Li, Wei Wang, Yang You,
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC 2021)
Honours and Awards
- [2024] A*STAR Computing & Information Science Scholarship
- [2021] Overall 2nd Place, ISC21 Student Cluster Competition
- [2020] Merit Award, APAC HPC-AI Competition
- [2018] NTU President Research Scholar
- [2017-2021] NTU Science & Technology Undergraduate Scholarship
- [2012-2016] Singapore SM1 Scholarship