Defense Date
2026
Document Type
Thesis
Degree Name
Master of Science in Dentistry
Department
Dentistry
First Advisor
Eser Tufekci
Second Advisor
Sarah Glass
Third Advisor
Caroline Carrico
Abstract
Objective: The primary aim was to evaluate whether large language models (LLMs) can generate differential diagnoses that include the true diagnosis when prompted with clinician-written lesion descriptions. A secondary aim was to determine whether clinician-written prompts after education in oral pathology descriptive terminology led to better AI-generated differentials. It was hypothesized that more detailed clinical descriptions would yield differentials containing the correct diagnosis.
Methods: Dentists at conferences and study clubs in Virginia participated in a survey on two adolescent cases of oral pathologies, mucoepidermoid carcinoma and osteosarcoma. Attendees were asked to describe lesions before and after a standardized lectureon oral pathology terminology and AI usage. Responses were anonymized, transcribed, and entered into two LLMs, namely ChatGPT and Copilot, to generate the top 10 differential diagnoses. AI memory controls were used to eliminate bias. McNemar’s chi-squared test and t-tests were used to analyze data, and the significance was set at α = 0.05.
Results: For Case 1, the correct diagnosis appeared in 45% (ChatGPT) and 43% (Copilot) of pre-lecture outputs, and in 42% and 52% of post-lecture outputs, respectively, with no significant differences between models or between pre- and post-lecturecomparisons (all p>0.20). The mean ranking of the correct diagnosis did not change significantly (ChatGPT: 5.1-5.3; Copilot: 5.3-6.3 out of 10). For Case 2, ChatGPT outperformed Copilot both pre- (29% vs 7%) (p ≤ 0.0006) and post-lecture (38% vs9%) (p ≤ 0.0001), with no significant pre- to post-lecture improvement for either model. Mean rankings of correct diagnosis were unchanged (ChatGPT 7.0-7.7 and Copilot 8.6-8.7 out of 10).
Conclusion: Both ChatGPT and Copilot failed to reliably generate differentials containing the true diagnosis when prompted with clinician-written lesion descriptions. Educational intervention on descriptive terminology did not improveAI-generated diagnostic accuracy. This study suggests that LLMs require further development before they can reliably assist with differential diagnoses in dentistry.
Rights
© The Author
Is Part Of
VCU University Archives
Is Part Of
VCU Theses and Dissertations
Date of Submission
6-18-2026