Web IR NLP Group @ NUS
Web IR NLP Group @ NUS
Tour
News
People
Events
Gallery
Publications
Projects
Contact
Light
Dark
Automatic
Efficiency
SSNLP 2026: WING Members Present Three Posters
SSNLP 2026: WING Members Present Three Posters
21 Jan, 2026
Singapore University of Technology and Design
Min-Yen Kan
,
Yisong Miao
,
Tongyao Zhu
,
Fanxiao Li
Context Window Scheduling for Efficient LLM Pretraining
Longer pretraining context windows are not always better. We study how to schedule the context window during LLM pretraining to balance long-context capability with training efficiency.
Tongyao Zhu
,
Qian Liu
,
Min-Yen Kan
Paper
arXiv
Code
Slides
Cite
×