BACKGROUND: The rising global elderly population increases the demand for caregiving, yet traditional methods may not fully assess the challenges faced by vital informal caregivers. OBJECTIVE: To investigate the efficacy of Large Language Model (LLM) in detecting overburdened informal caregivers, benchmarking against rule-based and machine learning methods. METHODS: 1,791 eligible informal caregivers from Southern Taiwan and utilized their textual case summary reports for the LLM. We also employed structured questionnaire results for machine learning models. Furthermore, we leveraged the visualization of the LLM's attention mechanisms to enhance our understanding of the model's interpretative capabilities. RESULTS: The LLM achieved an Area Under the Receiver Operating Characteristic (AUROC) curve of 0.84 and an Area Under the Precision-Recall Curve (AUPRC) of 0.70, marking an 8% and 14% improvement over traditional methods. The visualization of the attention mechanism accurately reflected the evaluations of human experts, concentrating on descriptions of high-burden descriptions and the relationships between caregivers and recipients. CONCLUSION: This research demonstrates the notable capability of LLM to accurately identify high-burden caregivers in Long-term Care (LTC) settings. Compared to traditional approaches, LLM offers an opportunity for the future of LTC research and policymaking.
Date:
2024-10
Relation:
Computer Methods and Programs in Biomedicine. 2024 Oct;255:Article number 108329.
