KL-Time Replay: Function-Space Drift Monitoring for Continual Learning in LLMs

Abstract

Replay-based continual learning for large language models requires deciding \emph{when} to revisit past examples. Recent work has shown that model-centric scheduling, which triggers replay based on accumulated parameter updates, outperforms fixed step-based approaches. However, parameter-space metrics may not directly reflect the behavioral changes that constitute forgetting. We propose KL-Time Replay, which monitors function-space drift by computing KL divergence between current and reference predictions on fixed anchor sets from previous tasks. When cumulative drift exceeds thresholds calibrated to an Ebbinghaus-inspired schedule, replay is triggered. On a 5-task text classification benchmark, KL-Time achieves comparable performance to FOREVER's parameter-space approach (OP 0.704 vs 0.708) while producing substantively different scheduling decisions (14.24% trigger divergence). Our analysis reveals that while KL drift and parameter update norms are highly correlated within tasks ($>$0.97), they diverge at task boundaries, establishing function-space drift as a viable alternative signal for replay scheduling.