Clinical Decision Support with Large Language Models: A Theoretical Framework for Safe Integration into Resource-Constrained Health Systems

Authors

  • Igwe Emmanuel Sunday Veristzon Global Research Lab Author
  • NWOGBOJI Emmanuel Usulor International Institute of AI and Research Author
  • ATOLAGBE Umar Kawu Veristzon Global Research Lab Author
  • OGAH Sunday Ogbo Veristzon Global Research Lab Author
  • BAMIGBOLA Paul Idowu Veristzon Global Research Lab Author

DOI:

https://doi.org/10.68854/kg8sas24

Keywords:

clinical decision support, large language models, patient safety, human oversight, bounded autonomy, primary health care, resource-constrained health systems, Africa

Abstract

Large language models (LLMs) can now summarise records, suggest differential diagnoses, support triage and converse with patients, and the health systems with the fewest clinicians face the strongest pressure to adopt them. Yet the evidence on their clinical use comes mostly from high-income settings, and existing guidance treats LLM-based clinical decision support as one undifferentiated technology rather than a set of tasks with different risks. This study aimed to specify where in the clinical workflow an LLM may safely sit in resource-constrained health systems, the failure modes characteristic of each position, and the safeguards each position requires. A theoretical, integrative theory-building design was used. The study population comprised peer-reviewed journal articles published from 2023 to 2026 on LLMs and tool-using agentic systems, clinical decision support, patient safety, governance, implementation and the health workforce in low-resource settings. Purposive, relevance-ranked sampling across eight search strings in PubMed/MEDLINE yielded 265 unique records; 213 were screened against predefined criteria, and 42 were retained, together with five landmark articles added by digital object identifier, giving a final sample of 47 articles. Every included article was verified against its Crossref publisher deposit, and none had been retracted. Data were extracted into a structured matrix and analysed by deductive and inductive thematic synthesis. Only 13 of the 47 articles were set in Africa, and only two reported prospective use of an LLM by African clinicians. The synthesis produced three outputs. The first is a taxonomy of four workflow positions of rising risk: documentation support, triage and referral support, differential diagnosis and management planning, and direct patient communication. The second is a five-domain safeguard architecture whose intensity scales with the risk of the position. The third is six testable propositions and seven design principles for bounded clinical autonomy. The central argument is that risk depends less on model capability than on interception distance, defined as the number of independent human checks between a model's output and the patient. The study recommends that health ministries, regulators and developers choose the workflow position before choosing the model, ground outputs in nationally adapted guidelines, default to referral under uncertainty, and grant greater autonomy only on the evidence of post-deployment monitoring.

References

Agweyu, A., Mwaniki, P., Menon, V., Korom, R., Isaaka, L., Wanyama, C., Gill, J., Kiptinness, S., Adan, N., Emmanuel-Fabula, M., Riley, R. D., Archer, L., Denniston, A. K., Liu, X., & Mateen, B. A. (2026). Generative AI-enabled clinical decision support system in primary care: A pragmatic, cluster-randomized trial. Nature Medicine, 32(8), 3032–3039. https://doi.org/10.1038/s41591-026-04503-6

Asgari, E., Montaña-Brown, N., Dubois, M., Khalil, S., Balloch, J., Yeung, J. A., & Pimenta, D. (2025). A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation. npj Digital Medicine, 8(1), Article 274. https://doi.org/10.1038/s41746-025-01670-7

Brodeur, P. G., Buckley, T. A., Kanjee, Z., Goh, E., Ling, E. B., Jain, P., Cabral, S., Abdulnour, R. E., Haimovich, A. D., Freed, J. A., Olson, A., Morgan, D. J., Hom, J., Gallo, R., McCoy, L. G., Mombini, H., Lucas, C., Fotoohi, M., Gwiazdon, M., . . . Rodman, A. (2026). Performance of a large language model on the reasoning tasks of a physician. Science, 392(6797), 524–527. https://doi.org/10.1126/science.adz4433

Cao, W., Zhang, Q., Liu, J., & Liu, S. (2026). From agents to governance: Essential AI skills for clinicians in the large language model era. Journal of Medical Internet Research, 28, Article e86550. https://doi.org/10.2196/86550

Chang, Y., Hsieh, M. H., Ju, P. C., Liu, Y. C., & Chang, C. C. (2026). Clinical plausibility in large language model robustness testing for medicine: A scoping review. Journal of Medical Systems, 50(1), Article 77. https://doi.org/10.1007/s10916-026-02405-1

CHART Collaborative, Huo, B., Collins, G. S., Chartash, D., Thirunavukarasu, A. J., Flanagin, A., Iorio, A., Cacciamani, G., Chen, X., Liu, N., Mathur, P., Chan, A. W., Laine, C., Pacella, D., Berkwits, M., Antoniou, S. A., Camaradou, J. C., Canfield, C., Mittelman, M., . . .

Guyatt, G. H. (2025). Reporting guideline for chatbot health advice studies: The CHART statement. JAMA Network Open, 8(8), Article e2530220. https://doi.org/10.1001/jamanetworkopen.2025.30220

Chen, H., Zeng, D., Qin, Y., Fan, Z., Ng Yu Ci, F., Klonoff, D. C., Ji, J. S., Zhang, S., Amissah-Arthur, K. N., Jiménez de Tavárez, M. M., Masood, S., Van Le, P., Keane, P. A., Sheng, B., Wong, T. Y., & Tham, Y. C. (2025). Large language models and global health equity: A roadmap for equitable adoption in LMICs. The Lancet Regional Health – Western Pacific, 63, Article 101707. https://doi.org/10.1016/j.lanwpc.2025.101707

Chuma, K. G., & Sibiya, P. T. (2026). Integrating ChatGPT to support children and adolescents with diabetes mellitus in South African healthcare: Perspectives of healthcare professionals and academics. African Journal of Primary Health Care & Family Medicine, 18(1), Article a5433. https://doi.org/10.4102/phcfm.v18i1.5433

Cornick, R. V., Picken, S. C., Awotiwon, A., Rabe, M., Wattrus, C., Fredericks, T., Timmerman, V., & Fairall, L. R. (2025). Addressing health information inequities: Making evidence-based clinical content more accessible in low- and middle-income primary care. BMJ Global Health, 9(Suppl 3), Article e013814. https://doi.org/10.1136/bmjgh-2023-013814

Ferber, D., El Nahhas, O. S. M., Wölflein, G., Wiest, I. C., Clusmann, J., Leßmann, M.-E., Foersch, S., Lammert, J., Tschochohei, M., Jäger, D., Salto-Tellez, M., Schultz, N., Truhn, D., & Kather, J. N. (2025). Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology. Nature Cancer, 6(8), 1337–1349. https://doi.org/10.1038/s43018-025-00991-6

Ferreira, J. C., & Rosa, I. (2026). Large language models in healthcare: Applications, evaluation frameworks, and governance pathways: A scoping review and multidimensional framework. Frontiers in Digital Health, 8, Article 1865568. https://doi.org/10.3389/fdgth.2026.1865568

Gaber, F., Shaik, M., Allega, F., Bilecz, A. J., Busch, F., Goon, K., Franke, V., & Akalin, A. (2025). Evaluating large language model workflows in clinical decision support for triage and referral and diagnosis. npj Digital Medicine, 8(1), Article 263. https://doi.org/10.1038/s41746-025-01684-1

Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Rodman, A., & Chen, J. H. (2024). Large language model influence on diagnostic reasoning: A randomized clinical trial. JAMA Network Open, 7(10), Article e2440969. https://doi.org/10.1001/jamanetworkopen.2024.40969

Guan, Z., Li, H., Hernández-Jiménez, S., Amissah-Arthur, K. N., Schmidt, M. I., Masood, S., Zeng, D., Jia, W., Lim, L. L., & Sheng, B. (2026). Can artificial intelligence bridge the gaps for primary diabetes care in low-income and middle-income countries? The Lancet Diabetes & Endocrinology, 14(5), 412–421. https://doi.org/10.1016/S2213-8587(26)00010-0

Gupta, S., Gal, Z. T., Touray, J., Luiselli, G. A., Ceesay, A., Manneh, E. K., Cham, M., Rolston, J. D., Chrenek, R., Bah, M. G., Golby, A. J., Esene, I. N., Arnaout, O., Sanchez, C., Janneh, L., Smith, T. R., & Jabang, J. N. (2026). Large language model for postoperative clinical decision support in a neurosurgery ward in the Gambia: A prospective pilot feasibility study. Neurosurgery, 99(5), 1127–1135. https://doi.org/10.1227/neu.0000000000003963

Hager, P., Jungmann, F., Holland, R., Bhagat, K., Hubrecht, I., Knauer, M., Vielhauer, J., Makowski, M., Braren, R., Kaissis, G., & Rueckert, D. (2024). Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine, 30(9), 2613– 2622. https://doi.org/10.1038/s41591-024-03097-1

Hasan, M. M., Hossain, M. A., Sayem, F. H., Paul, B. K., Rahman, Z., Uddin, M. S., & Mostafiz, R. (2026). CLIN-LLM: A safety-constrained hybrid framework for clinical diagnosis and treatment generation. PLOS ONE, 21(8), Article e0348611. https://doi.org/10.1371/journal.pone.0348611

Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., Sanger, S., Ramji, K., & Guyatt, G. (2025). Large language models for chatbot health advice studies: A systematic review. JAMA Network Open, 8(2), Article e2457879. https://doi.org/10.1001/jamanetworkopen.2024.57879

Hussein, A. M., & Yusuf, F. Y. (2026). Artificial intelligence and mental health services in Somalia: Opportunities and health-system considerations. Journal of Multidisciplinary Healthcare, 19, 1–4. https://doi.org/10.2147/JMDH.S622655

Kimera, R., Kuyeso, R., Maleka, E., Tukamushaba, F., William, W., Mwavu, R., Bamutura, D. S., Ngonzi, J., Ainomugisha, B., Namuli, A., Namiiro, P., Nyakato, C., Celi, L. A., & Kaggwa, F. (2026). Dialogues of delivery: A multilingual question-answer dataset for maternal healthcare in East African languages. BMC Research Notes, 19(1), Article 317. https://doi.org/10.1186/s13104-026-07899-z

Li, J., Guan, Z., Wang, J., Cheung, C. Y., Zheng, Y., Lim, L. L., Lim, C. C., Ruamviboonsuk, P., Raman, R., Corsino, L., Echouffo-Tcheugui, J. B., Luk, A. O. Y., Chen, L. J., Sun, X., Hamzah, H., Wu, Q., Wang, X., Liu, R., Wang, Y. X., . . . Wong, T. Y. (2024). Integrated image-based deep learning and language models for primary diabetes care. Nature Medicine, 30(10), 2886–2896. https://doi.org/10.1038/s41591-024-03139-8

Lima, H. A., Trocoli-Couto, P. H. F. S., Moazzam, Z., Rocha, L. C. D., Pagano, A., Martins, F. F., Brabo, L. T., Reis, Z. S. N., Keder, L., Begum, A., Mamede, M. H., Pawlik, T. M., & Resende, V. (2025). Quality assessment of large language models' output in maternal health. Scientific Reports, 15(1), Article 22474. https://doi.org/10.1038/s41598-025-03501-x

Liu, S., Peterson, J. F., & Wright, A. (2026). How generative AI should transform clinical decision support. JAMA, 336(7), 543–546. https://doi.org/10.1001/jama.2026.13187

Low, Y. S., Jackson, M. L., Hyde, R. J., Brown, R. E., Sanghavi, N. M., Baldwin, J. D., Pike, C. W., Muralidharan, J., Hui, G., Alexander, N., Hassan, H., Nene, R. V., Pike, M., Pokrzywa, C. J., Vedak, S., Yan, A. P., Yao, D. H., Zipursky, A. R., Dinh, C., . . . Gombar, S. (2025). Answering real-world clinical questions using large language model, retrieval-augmented generation, and agentic systems. Digital Health, 11, Article 20552076251348850. https://doi.org/10.1177/20552076251348850

Mahajan, A., Obermeyer, Z., Daneshjou, R., Lester, J., & Powell, D. (2025). Cognitive bias in clinical large language models. npj Digital Medicine, 8(1), Article 428. https://doi.org/10.1038/s41746-025-01790-0

Malpani, R., & Moodley, K. (2025). Large multimodal models: Boon or burden for low- and middle-income countries. International Health, 17(5), 853–855. https://doi.org/10.1093/inthealth/ihae073

Menon, V., Shimelash, N., Rutunda, S., Nshimiyimana, C., Archer, L., Emmanuel-Fabula, M., Berhe, D. F., Gill, J., Hezagira, E., Remera, E., Riley, R., Wong, R., Denniston, A. K., Mateen, B. A., & Liu, X. (2025). Assessing the potential utility of large language models for assisting community health workers: Protocol for a prospective, observational study in Rwanda. BMJ Open, 15(10), Article e110927. https://doi.org/10.1136/bmjopen-2025-Meskó, B., & Topol, E. J. (2023). The imperative for regulatory oversight of large language models (or generative AI) in healthcare. npj Digital Medicine, 6(1), Article 120. https://doi.org/10.1038/s41746-023-00873-0 Mourão, C. F., & Juliasse, L. E. (2026). Accountability for large language models in health care. Bulletin of the World Health Organization, 104(10), 698–704. https://doi.org/10.2471/BLT.26.296186

Muliokela, R. K., Banda, K., Hussen, A. M., Malumo, S. B., Kashoka, A., Mwiche, A., Chiboma, I., Barreix, M., Nyirenda, M., Sithole, Z., Ratanaprayul, N., Endehabtu, B. F., Telake, H. A., Weldeab, A., Probert, W. J. M., Tunçalp, Ö., Maya, E., Woldetsadik, M., Tilahun, B., . . .

Tamrat, T. (2025). Implementation of WHO SMART guidelines-digital adaptation kits in pathfinder countries in Africa: Processes and early lessons learned. JMIR Medical Informatics, 13, Article e58858. https://doi.org/10.2196/58858

Okoroafor, S. C., & Dela Christmals, C. (2024). Health professions education strategies for enhancing capacity for task-shifting and task-sharing implementation in Africa: A scoping review. Journal of Continuing Education in the Health Professions, 44(3), 194–202. https://doi.org/10.1097/CEH.0000000000000517

Ong, J. C. L., Jin, L., Elangovan, K., Lim, G. Y. S., Lim, D. Y. Z., Sng, G. G. R., Ke, Y. H., Tung, J. Y. M., Zhong, R. J., Koh, C. M. Y., Lee, K. Z. H., Chen, X., Ch'ng, J. K., Than, A., Goh, K. J., Lim, C. P., Ng, T. M., Liu, N., & Ting, D. S. W. (2025). Large language model as clinical decision support system augments medication safety in 16 clinical specialties. Cell Reports Medicine, 6(10), Article 102323. https://doi.org/10.1016/j.xcrm.2025.102323

Osonuga, A., Osonuga, A. A., Fidelis, S. C., Osonuga, G. C., Juckes, J., & Olawade, D. B. (2025). Bridging the digital divide: Artificial intelligence as a catalyst for health equity in primary care settings. International Journal of Medical Informatics, 204, Article 106051. https://doi.org/10.1016/j.ijmedinf.2025.106051

Peiffer-Smadja, N., Descousse, S., Courrèges, E., Nganbou, A., Jeanmougin, P., Birgand, G., Lénaud, S., Beaumont, A.-L., Durand, C., Delory, T., Le Bel, J., Bouvet, E., Lariven, S., D'Ortenzio, E., Konaté, I., Bouyou-Akotet, M. K., Ouedraogo, A.-S., Kouakou, G. A., Poda, A., . . .

Tanon, A. (2024). Implementation of a clinical decision support system for antimicrobial prescribing in sub-Saharan Africa: Multisectoral qualitative study. Journal of Medical Internet Research, 26, Article e45122. https://doi.org/10.2196/45122

Pieterse, P. (2025). Health workers in sub-Saharan Africa: Concurrent skilled health worker shortages and under-employment. The International Journal of Health Planning and Management, 40(6), 1335–1340. https://doi.org/10.1002/hpm.70001

Rath, S., Barzani, S., Okesanya, O. J., Tawfik, A. M., Shehada, W., Aziz, M. M., Urbon, S., & Alsabri, M. (2025). Optimizing pediatric emergency triage in low-resource settings: Evidence-based strategies, task-shifting, and technological innovations. International Journal of Emergency Medicine, 18(1), Article 216. https://doi.org/10.1186/s12245-025-00998-x

Sandmann, S., Hegselmann, S., Fujarski, M., Bickmann, L., Wild, B., Eils, R., & Varghese, J. (2025). Benchmark evaluation of DeepSeek large language models in clinical decision-making. Nature Medicine, 31(8), 2546–2549. https://doi.org/10.1038/s41591-025-03727-2

Scipion, C. E. A., Manchester, M. A., Federman, A., Wang, Y., & Arias, J. J. (2025). Barriers to and facilitators of clinician acceptance and use of artificial intelligence in healthcare settings: A scoping review. BMJ Open, 15(4), Article e092624. https://doi.org/10.1136/bmjopen-2024-

Sheffel, A., Andrews, K. G., Conner, R., Di Giorgio, L., Evans, D. K., Gatti, R., Lindelow, M., Sharma, J., Svensson, J., Wane, W., & Welander Tärneberg, A. (2024). Human resource challenges in health systems: Evidence from 10 African countries. Health Policy and Planning, 39(7), 693–709. https://doi.org/10.1093/heapol/czae034

Shidende, N., & Mwogosi, A. (2025). Exploring the impact of generative AI tools on healthcare delivery in Tanzania. Journal of Health Organization and Management, 39(8), 1580–1599. https://doi.org/10.1108/JHOM-01-2025-0007

Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Seneviratne, M., Gamble, P., Kelly, C., Babiker, A., Schärli, N., Chowdhery, A., Mansfield, P., Demner-Fushman, D., . . . Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180. https://doi.org/10.1038/s41586-023-06291-2

Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., Xu, B., Zhang, Q., Zhao, J., Li, Y., & Li, H. (2025). Large language models in medical diagnostics: Scoping review with bibliometric analysis. Journal of Medical Internet Research, 27, Article e72062. https://doi.org/10.2196/72062

Tegenaw, G. S., Sori, D. A., Teklemariam, G. K., Verbeke, F., Cornelis, J., & Jansen, B. (2024). Evaluation of a computer-aided clinical decision support system for point-of-care use in low-resource primary care settings: Acceptability evaluation study. JMIR Human Factors, 11, Article e47631. https://doi.org/10.2196/47631

Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023). Large language models in medicine. Nature Medicine, 29(8), 1930–1940. https://doi.org/10.1038/s41591-023-02448-8

Weissman, G. E., Mankowitz, T., & Kanter, G. P. (2025). Unregulated large language models produce medical device-like output. npj Digital Medicine, 8(1), Article 148. https://doi.org/10.1038/s41746-025-01544-y

Yu, L., Darmstadt, G. L., Ward, V., Wong, R. J., Stevenson, D. K., & Maric, I. (2026). Large language models for maternal and neonatal health care in low- and middle-income countries. The Journal of Pediatrics, 293, Article 115037. https://doi.org/10.1016/j.jpeds.2026.115037

Zhao, W., Wu, C., Fan, Y., Qiu, P., Zhang, X., Sun, Y., Zhou, X., Zhang, S., Peng, Y., Wang, Y., Sun, X., Zhang, Y., Yu, Y., Sun, K., & Xie, W. (2026). An agentic system for rare disease diagnosis with traceable reasoning. Nature, 651(8106), 775–784. https://doi.org/10.1038/s41586-025-10097-9

Published

2026-10-27