vSFT: Learning to Verbally Tune Agents at Test Time
Qishuai Wen, Rui Chen, Haiwen Li, Zhiyuan Huang, Qiwen Gu, Bingjie Gao, Geng Li, Jing Tang, Chu Xiangxiang, Chun-Guang Li
Under review
We propose vSFT, a test-time adaptation method for agents that provides a learnable, effective self-improvement mechanism by iteratively internalizing verbal feedback in a closed loop.
vSFT: Learning to Verbally Tune Agents at Test Time
Qishuai Wen, Rui Chen, Haiwen Li, Zhiyuan Huang, Qiwen Gu, Bingjie Gao, Geng Li, Jing Tang, Chu Xiangxiang, Chun-Guang Li
Under review
We propose vSFT, a test-time adaptation method for agents that provides a learnable, effective self-improvement mechanism by iteratively internalizing verbal feedback in a closed loop.
Mixture-of-Top-k Attention: Efficient Attention via Scalable Fast Weights
Qishuai Wen, Zhiyuan Huang, Xianghan Meng, Wei He, Chun-Guang Li
NeurIPS 2026
We introduce a fast-weight perspective that unifies existing efficient attention designs, thereby revealing the limitations of prior methods and addressing them with our proposed Mixture-of-Top-k Attention (MiTA).
Mixture-of-Top-k Attention: Efficient Attention via Scalable Fast Weights
Qishuai Wen, Zhiyuan Huang, Xianghan Meng, Wei He, Chun-Guang Li
NeurIPS 2026
We introduce a fast-weight perspective that unifies existing efficient attention designs, thereby revealing the limitations of prior methods and addressing them with our proposed Mixture-of-Top-k Attention (MiTA).
Towards Interpretable and Efficient Attention: Compressing All by Contracting a Few
Qishuai Wen, Zhiyuan Huang, Chun-Guang Li
NeurIPS 2025 Spotlight
We propose an interpretable and efficient attention mechanism, Contract-and-Broadcast Self-Attention (CBSA), which also unifies a wide range of existing methods, including channel attention, linear attention, and full attention.
Towards Interpretable and Efficient Attention: Compressing All by Contracting a Few
Qishuai Wen, Zhiyuan Huang, Chun-Guang Li
NeurIPS 2025 Spotlight
We propose an interpretable and efficient attention mechanism, Contract-and-Broadcast Self-Attention (CBSA), which also unifies a wide range of existing methods, including channel attention, linear attention, and full attention.
Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective
Qishuai Wen, Chun-Guang Li
NeurIPS 2024
We establish connections between compression and semantic segmentation, particularly between principal component analysis and Transformer decoders. Based on this perspective, we derive DEPICT, a white-box, fully attentional decoder that is lightweight, interpretable, and robust.
Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective
Qishuai Wen, Chun-Guang Li
NeurIPS 2024
We establish connections between compression and semantic segmentation, particularly between principal component analysis and Transformer decoders. Based on this perspective, we derive DEPICT, a white-box, fully attentional decoder that is lightweight, interpretable, and robust.