Medical AI decision support systems' usability assessment through the lens of an expert's perspective in a preliminary investigation
In a recent study, experts focused more attention on user interface elements related to AI support when it was in use, as more frequent transitions from the image itself to user interface elements were observed. The study, which compared standard viewing of bitewing images without AI support to viewing where AI support could be toggled on and off, aimed to design AI systems with the user in mind, using eye tracking metrics for usability evaluation.
The findings suggest that the use of AI may impact the flow and efficiency of the inspection process, as increased time might be needed for overall assessment due to interruptions in attention caused by AI usage. However, the study did not find any significant difference in the detection of caries between standard viewing and viewing with AI support.
The study's results indicate that the usability of AI for caries detection in bitewing x-ray images is a complex issue, requiring further research and development. Key points supported by recent research and frameworks include:
- Validity and Clinical Appropriateness: AI models should be assessed with relevant performance metrics specific to the clinical task, such as sensitivity and specificity for identifying caries on bitewing x-rays. Validation studies must confirm that AI performance translates well to real-world clinical settings rather than solely laboratory conditions.
- Usefulness and Workflow Integration: Successful systems are integrated seamlessly into radiologists’ or dentists’ workflows, minimizing extra cognitive load or disruption. Eye tracking studies during caries detection can reveal how AI support alters visual search behavior — ideally focusing clinician attention on diagnostically relevant regions, reducing missed lesions and unnecessary re-examinations. Interfaces should reduce time demands, simplify interaction, and provide timely, context-sensitive alerts or suggestions so that clinicians experience net efficiency gains.
- Transparency and Explainability: User trust is bolstered by AI systems that provide clear explanations of their outputs, especially highlighting uncertainties. Providing visual overlays or heatmaps aligned with eye tracking data can help clinicians understand why AI flags specific areas in bitewing images, bridging the gap between black-box AI outputs and human interpretability.
- Accessibility and Usability: UI design should account for real-world usage conditions such as screen quality, lighting, and user visual abilities. Design improvements such as high contrast, legible fonts, and non-color cues support users including those with color blindness or visual impairment. Eye tracking can identify UI elements that draw or distract attention, guiding iterative refinement for user-friendliness and reducing cognitive fatigue.
- Clinician Oversight and Legal Compliance: The AI should augment rather than replace clinician judgment. Systems must be auditable and compliant with local regulations to ensure safety and accountability, important in sensitive uses like caries detection which influence treatment decisions.
- Continuous Monitoring and Feedback: Eye tracking metrics serve as a powerful tool for measuring how clinicians interact with AI-supported imaging systems. Tracking fixation times, saccades, and gaze patterns on x-ray images helps developers optimize AI interface design by confirming whether AI outputs focus clinician attention effectively and reduce scanning errors or overlook.
In conclusion, the best practices for designing user-friendly medical AI support systems, particularly in contexts like caries detection from bitewing x-ray images, emphasize validation in clinical context, ease of use via intuitive interfaces, interpretability, and integration into existing workflows, with eye tracking metrics providing objective feedback on usability and diagnostic attention patterns. This approach not only enhances diagnostic accuracy in caries detection from bitewing radiographs but also boosts clinician efficiency and trust in AI tools employed in healthcare.
- The integration of eye tracking technology into medical AI systems for caries detection in bitewing x-ray images could potentially improve diagnostic accuracy by focusing clinician attention on diagnostically relevant regions, thereby reducing the chances of missed lesions and the need for unnecessary re-examinations.
- Developers of medical AI systems for health and wellness should consider the use of technology such as eye tracking during usability evaluation, as it can provide valuable insights into how the systems are being used, helping to optimize interfaces, improve transparency, and increase user trust and efficiency in the detection of medical-conditions like caries.
