Counterfactual Gate Supervision Does Not Fix Gating Credit Assignment in Engram-Style Conditional Memory

Abstract

Engram-style conditional memory augments transformers with hash-indexed n-gram embeddings and learned gating, but prior work has identified a critical training pathology: gates become systematically mis-calibrated, preferring high-frequency ``hot'' memory slots even when low-frequency ``cold'' positions achieve lower loss. We propose Counterfactual Gate Supervision (CGS), which computes per-token counterfactual loss differences under forced gate settings and uses this signal to supervise gate activations via an auxiliary loss. Despite its principled motivation, our experiments reveal that CGS fails to fix gating credit assignment: oracle-AUC degrades from 0.549 to 0.528, and the hot/cold flip pathology persists. An iso-compute control demonstrates that CGS's modest validation loss improvement (4.467 vs 4.481) is fully attributable to extra computation from forced forward passes, not improved supervision---the iso-compute baseline achieves even better loss (4.452) without oracle-AUC degradation. This negative result suggests that per-token counterfactual signals are too noisy to provide useful gate supervision at this scale.