About Me

I am a master’s student at Tsinghua University, Shenzhen International Graduate School, majoring in Electronic Information. My research focuses on LLM compression and acceleration, as well as attention mechanism optimization. I am particularly interested in developing efficient and robust techniques for deploying outlier-free transformer models under limited computational resources. I have conducted research internships at Huawei Technologies and Meituan, where I worked on projects related to KV cache quantization, MoE LLM compression, and systematic outliers in LLMs. These experiences have strengthened my understanding of both the theoretical foundations of model compression and the practical aspects of system-level optimization for deploying transformer models.

Publications

• Longcat-flash: Technical Report
  MLC Team, B. Li, B. Lei, B. Wang, B. Rong, C. Wang, C. Zhang, C. Gao, … Z. Su

• KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
  Z. Su, K. Yuan
  Published at COLM 2025, 2025.

• Unveiling Super Experts in Mixture-of-Experts Large Language Models
  Z. Su, Q. Li, H. Zhang, Y. L. Qian, Y. Xie, K. Yuan

• AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models
  Z. Su, W. Shen, L. Li, Z. Chen, H. Wei, H. Yu, K. Yuan
  Published at ICME 2025, 2025.

• RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
  Z. Su, Z. Chen, W. Shen, H. Wei, L. Li, H. Yu, K. Yuan
  Published at IJCAI 2025, 2025.

Work Experience

• Huawei Technologies, Research Intern — 06.2024 – 02.2025
  Worked on LLM and VLM quantization, KV cache compression, and the study of systematic outliers in LLMs.
  Recognized with the Outstanding Practice Project Award (Top 10% of interns).

• Meituan, Research Intern — 03.2025 – 08.2025
  Focused on MoE LLMs quantization and research on super-expert mechanisms for MoE LLMs.
  Received an offer from the Meituan Beidou Program for LLMs talents.

Education

• M.S., Electronic Information — Tsinghua University, 2023 – 2026
  Awarded the First-Class Scholarship for outstanding academic performance.

Honors and Awards

• 🏆 Huawei Research Excellence Award, 2024
  For outstanding contributions to LLM and VLM quantization, KV cache compression, and research on systematic outliers.

• 🎓 First-Class Scholarship, Tsinghua University, Shenzhen International Graduate School, 2025
  Awarded for excellent academic performance during the master’s program.

• 💡 Meituan Beidou Program Offer, 2025
  Selected for the Beidou talent program for LLMs research based on internship performance.