Research Assistant Professor · Shenzhen Loop Area Institute
Congliang Chen
I am a Research Assistant Professor at the Shenzhen Loop Area Institute, affiliated with the Center for AI Theoretical Foundation and Systems. I received my B.S. from the School of Electronics Engineering and Computer Science, Peking University, and my Ph.D. from The Chinese University of Hong Kong, Shenzhen, advised by Prof. Zhi-Quan (Tom) Luo.
My research focuses on optimization algorithms and machine learning systems for modern AI models. My broader research goal is to build core methodological capabilities for large-model agents from mathematical structure and theoretical analysis, and to turn those capabilities into verifiable performance gains for real training and inference workloads.
I am especially interested in structured quasi-Newton and second-order optimization, generalization-oriented training algorithms, post-training and alignment optimization, distributed optimization, optimization methods for reinforcement learning, low-precision training and inference, model adaptation and parameter-efficient tuning, and high-performance operator generation, benchmarking, and performance modeling.
Recruiting
I am recruiting Research Assistants, prospective PhD students, and motivated interns in two directions: optimization algorithms and machine learning systems.
Please email chencongliang@slai.edu.cn with your CV, transcript if available, a short research/engineering summary, and links to papers, code, or projects. PhD applicants should also follow official SLAI admissions information, including the International Admissions Information page when applicable.
Open directions and student fit
Optimization Algorithms
- Structured quasi-Newton and second-order algorithms for large-scale learning and adaptation.
- Generalization-oriented optimization algorithms and theory-informed training principles.
- Post-training and alignment optimization algorithms.
- Distributed optimization algorithms for multi-worker and heterogeneous training environments.
- Optimization methods for reinforcement learning and agent training.
Machine Learning Systems
- Low-precision training and inference systems.
- Model adaptation, parameter-efficient tuning, and deployment-oriented fine-tuning systems.
- High-performance operator generation, benchmarking, and performance modeling for modern AI workloads.
Good fit: strong math foundations, solid Python/deep-learning programming, and interest in both theory and real-model experiments. CUDA/Triton/systems experience is a plus.
News
- 2026 We are looking for research assistants, prospective PhD students, and interns in optimization algorithms and machine learning systems.
- 2025 Adam-mini and GEM for diversity-preserving SFT were accepted to ICLR 2025.
- 2024 Why Transformers Need Adam appeared at NeurIPS 2024.
Research Interests
- Optimization algorithms: structured quasi-Newton and second-order methods, generalization-oriented optimization, post-training and alignment optimization, distributed optimization, and reinforcement-learning optimization.
- Machine learning systems: low-precision training and inference, model adaptation and parameter-efficient tuning, and high-performance operator generation, benchmarking, and performance modeling.
Selected Publications
(* indicates equal contribution, † indicates corresponding author).
Adam-mini: Use Fewer Learning Rates To Gain More
Yushun Zhang*, Congliang Chen*, Ziniu Li, Tian Ding, Chenwei Wu, Diederik P. Kingma, Yinyu Ye, Zhi-Quan Luo, and Ruoyu Sun. ICLR 2025.
Preserving Diversity in Supervised Fine-Tuning of Large Language Models
Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Zhi-Quan Luo, and Ruoyu Sun. ICLR 2025.
Why Transformers Need Adam: A Hessian Perspective
Yushun Zhang, Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, and Zhi-Quan Luo. NeurIPS 2024.
Efficient-Adam: Communication-Efficient Distributed Adam
Congliang Chen, Li Shen, Wei Liu, and Zhi-Quan Luo. IEEE Transactions on Signal Processing, 2023.
Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration
Congliang Chen*, Li Shen*, Fangyu Zou*, and Wei Liu. JMLR 2022.
Show more publications
- Quantized Adam with Error Feedback
Congliang Chen, Li Shen, Haozhi Huang, and Wei Liu. ACM Transactions on Intelligent Systems and Technology, 2021. - A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration
Li Shen, Congliang Chen, Fangyu Zou, Zequn Jie, Ju Sun, and Wei Liu. IEEE Transactions on Neural Networks and Learning Systems, 2023. - Adam Can Converge Without Any Modification On Update Rules
Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, and Zhi-Quan Luo. NeurIPS 2022. - Communication Efficient Primal-Dual Algorithm for Nonconvex Nonsmooth Distributed Optimization
Congliang Chen, Jiawei Zhang, Li Shen, Peilin Zhao, and Zhi-Quan Luo. AISTATS 2021.
Experience
Education
- 2018.08 - 2025.03, Ph.D., The Chinese University of Hong Kong, Shenzhen.
- 2014.09 - 2018.06, B.S., Peking University.
Experience & Service
- 2025 - Present, Research Assistant Professor, Shenzhen Loop Area Institute.
- 2019.07 - 2023.07, Research Intern, Tencent AI Lab, Shenzhen.
- Reviewer for ICML, NeurIPS, ICLR, ICCV, CVPR, and related venues.