Evidence-Grounded Constraint Schemas Do Not Improve Medical LLM Guardrails on LiveMedBench

Abstract

Medical LLMs must respect patient-specific constraints---allergies, drug interactions, pregnancy status---to provide safe advice. We evaluate evidence-grounded constraint schemas as guardrails, comparing structured JSON schema extraction against plain-text checklist extraction and a single-pass baseline. On 500 constraint-salient cases from LiveMedBench, neither guardrail approach improves over the baseline: the structured schema scores 0.522 versus baseline 0.535 on constraint-focused rubric ($\Delta=-0.013$), while the checklist scores 0.512 ($\Delta=-0.024$). Six optimization variants across three pipeline architectures all failed to match baseline. Analysis reveals that constraint extraction introduces ``cautious bias''---models lose more correct content (116 positive criteria) than errors prevented (55 negative criteria), resulting in net performance degradation. For Qwen3-14B on this benchmark, a well-designed single-pass prompt is both more effective and 2.4--2.7$\times$ more efficient than multi-pass guardrail pipelines.