Anisotropic Noise Fingerprints Reveal Concept Choice in Concept-Aware Embedding Privacy

Abstract

Concept-aware embedding sanitization applies learned, concept-specific anisotropic noise to protect privacy while preserving utility. We show that under multi-release access---where an attacker observes multiple sanitized versions of the same document---the anisotropic noise structure leaks which privacy concept was selected. Our variance fingerprint attack computes per-dimension variance profiles from embedding differences and matches them to concept templates, achieving 100% concept-identification accuracy on SPARSE-style sanitization, compared to 18.9% for isotropic noise (near the 20% chance level for $K=5$ concepts). Covariance smoothing fails to mitigate the attack: even at $\lambda=0.99$ (mixing 99% identity covariance), accuracy remains 63.3%. Importantly, this is metadata leakage about the privacy mode, not content leakage---token-level privacy is preserved across all conditions (AUC $\sim$0.52 vs 0.87 for clean embeddings). Our findings reveal a fundamental limitation of concept-aware anisotropic noise under realistic multi-release threat models.