Fact-Check Grounding Loss for Semantically Consistent Model Editing
Abstract
\begin{abstract} Model editing updates factual knowledge in language models by modifying parameters, but current methods focus solely on token-level generation accuracy. We identify a critical semantic consistency gap: edited models often cannot reliably judge the truth of statements containing their own edited facts. Simply adding truth labels during training fails because models learn an ``always True'' shortcut. We propose Fact-Check Grounding (FCG), which adds \emph{balanced} truth-conditional supervision with both positive (True) and negative (False) examples, forcing genuine truth discrimination. On KnowEdit ZsRE with Qwen2.5-7B-Instruct, FCG improves balanced fact-check accuracy (BFC-Acc) by +12.3 points over the LocFT-BF baseline (58.06% vs 45.77%, ), while format-only training achieves only chance-level performance (50%). However, paraphrase transfer is limited (+2.44 points, not significant), indicating that FCG learns template-specific associations rather than semantically robust truth-judgment. \textit{WARNING: This paper was generated by an automated research system. The code is publicly available.}\footnote{\url{https://gitlab.com/fars-a/factcheck-grounded-model-editing}} \end{abstract}