Case Mix and Protocol Constraints in Diagnostic Language Model Conversations: A Secondary Analysis of Released MedR-Bench Outputs

Introduction Diagnostic language models can request additional examinations before answering. Their observed stopping behavior depends on the model, the information returned by a simulator, and the interaction protocol, and these contributions are difficult to separate in aggregate benchmark results. We quantified an explicit unavailable-information text marker in simulator feedback and its association with fifth-response contact, a protocol-imposed interaction limit. Methods We analyzed released MedR-Bench diagnosis conversations from seven model labels and 957 source case reports. Exposure was a text marker: the exact simulator sentence stating that no relevant ancillary test information was available, in the first simulator feedback. It was not a clinical judgment that information was missing. The outcome was fifth-response contact, meaning that the run reached the fifth assistant response. Within-model risk differences were averaged with equal model weights, with 5,000 report-cluster bootstrap resamples. Sensitivity analyses used linear probability models with model effects and then report and model effects, with and without first-request composition, and leave-one-model-out summaries. Results Of 6,699 expected report-model cells, 6,690 were released, and 5,719 entered the first-feedback analysis; entry ranged from 41.2% to 98.0% of released runs by model. The marker appeared in 4,593 first feedbacks (80.3%), and 96.2% of these also returned other text for at least one requested item. All 1,001 fifth-response runs followed a mandatory-final-answer prompt. The equal-model-weighted difference was 6.37 percentage points (95% CI 4.16 to 8.54), or 4.16 points without the model with the largest difference. The difference largely reflected case composition: the common-slope coefficient was 5.61 points with model effects, 1.05 points (95% CI -1.64 to 3.74) with report and model effects, and 1.21 points (-1.67 to 4.08) after adding request composition. In paired one-round and free-turn releases, marker status differed in 1,309 of 5,701 pairs (23.0%) despite identical case text. Conclusions The positive association between the unavailable-information text marker and fifth-response contact was largely explained by case composition and was close to null within source reports. Case composition and protocol states should remain visible when diagnostic trajectories are evaluated. These data do not establish a causal effect of missing information or support heart-failure clinical validation.