FROM ALGORITHMS TO EVIDENCE: ASSESSING ARTIFICIAL INTELLIGENCE CHATBOTS AND DRUG DATABASES FOR DETECTING CARDIO-DIABETIC DRUG INTERACTIONS

AFTAB ALAM; ANUKRITI SARAN; RADHIKA JOSHI; SATHVIK B. SRIDHAR; SWAPNIL SHARMA; SARVESH PALIWAL

doi:10.22159/ijap.2025v17i5.54398

Authors

AFTAB ALAM Department of Pharmacy, Banasthali Vidyapith, Rajasthan-304022, India. Department of Clinical Pharmacy and Pharmacology, RAK College of Pharmacy, RAK Medical and Health Sciences University, Ras Al Khaimah-11172, United Arab Emirates https://orcid.org/0000-0002-6020-9290
ANUKRITI SARAN Department of Bioscience and Biotechnology, Banasthali Vidyapith, Rajasthan-304022, India https://orcid.org/0000-0002-9076-7576
RADHIKA JOSHI Department of Pharmacy, Banasthali Vidyapith, Rajasthan-304022, India. Department of Bioscience and Biotechnology, Banasthali Vidyapith, Rajasthan-304022, India https://orcid.org/0009-0001-8064-7036
SATHVIK B. SRIDHAR Department of Clinical Pharmacy and Pharmacology, RAK College of Pharmacy, RAK Medical and Health Sciences University, Ras Al Khaimah-11172, United Arab Emirates
SWAPNIL SHARMA Department of Pharmacy, Banasthali Vidyapith, Rajasthan-304022, India https://orcid.org/0000-0003-2639-7096
SARVESH PALIWAL Department of Pharmacy, Banasthali Vidyapith, Rajasthan-304022, India

DOI:

https://doi.org/10.22159/ijap.2025v17i5.54398

Keywords:

Artificial intelligence, Drug interactions, patient safety, Drug databases, Drug information resources

Abstract

Objective: Electronic drug information resources are widely accessible and commonly used by healthcare professionals for identifying drug-drug interactions (DDIs). With the rapid advancements in artificial intelligence (AI), AI-powered chatbots have demonstrated their potential in detecting DDIs. However, variations exist in the scope, completeness, and consistency of information provided by different resources. This study aims to conduct a comparative evaluation of drug interaction databases and AI chatbots to assess their reliability in DDI identification.

Methods: A total of three databases, namely Lexicomp, Drugs.com, DrugBank and AI-powered chatbots such as ChatGPT, Copilot and Gemini were used for comparative evaluation. The percentage of interactions that had an entry in each drug information resource was used to score each resource for scope. For each resource that described clinical effects, severity, mechanism, clinical management, and risk factors, a completeness score was calculated. The consistency of the information was assessed using the Fleiss' Kappa (κ) score, estimated with the Statistical Package for the Social Sciences (SPSS), version 29.0 (IBM, USA).

Results: A total of 150 drug pairs were selected in the present study. The scope score was highest (100%) for Lexicomp, ChatGPT and Gemini. The completeness score was highest (100%) in all the AI-powered chatbots, followed by Drugs.com (90%) and Lexicomp (85.2%). Fleiss' kappa coefficient was used to determine the inter-resource agreement on DDI severity classification and the overall agreement was categorized as fair (κ=0.28, p<0.001). Cohen’s kappa coefficients were calculated to evaluate pairwise agreement among the resources and the overall mean kappa coefficient (κ=0.51, p<0.01) indicated a moderate level of agreement among the resources.

Conclusion: Significant differences amongst the resources were observed in terms of severity classification. Using Lexicomp as reference, accuracy assessment was done and variable sensitivity, specificity, and predictive values among resources were observed. A moderate overall agreement in the inter-resource agreement on DDI presence-absence, with traditional databases showed stronger pairwise agreement than AI chatbots.

References

1. Corrigendum to: facing the challenge of polypharmacy when prescribing for older people with cardiovascular disease. A re-view by the European Society of Cardiology Working Group on cardiovascular pharmacotherapy. Eur Heart J Cardiovasc Phar-macother Eur Heart J Cardiovasc Pharmacother. 2023;9(3):291. doi: 10.1093/ehjcvp/pvad013, PMID 36786239.

2. World Health Organization. Hypertension. In: Geneva: World Health Organization; 2023 Mar 16. Available from: https://www.who.int/news-room/fact-sheets/detail/hypertension. [Last accessed on 10 Feb 2025].

3. Soomro MH, Jabbar A. Diabetes etiopathology classification, diagnosis and epidemiology. In: Shera AS, Jawad F, editors. BIDE’s diabetes desk book. 1st ed. Amsterdam: Elsevier; 2024. p. 19-42. doi: 10.1016/B978-0-443-22106-4.00022-X.

4. Wang Z, Yang T, Fu H. Prevalence of diabetes and hypertension and their interaction effects on cardio cerebrovascular diseases: a cross-sectional study. BMC Public Health. 2021 Jun 25;21(1):1224. doi: 10.1186/s12889-021-11122-y, PMID 34172039, PMCID PMC8229421.

5. Tatsumi Y, Ohkubo T. Hypertension with diabetes mellitus: significance from an epidemiological perspective for Japanese. Hypertens Res. 2017 Sep;40(9):795-806. doi: 10.1038/hr.2017.67, PMID 28701739.

6. Lati MS, David NG, Kinuthia RN. The predictors of potential drug-drug interactions among diabetic hypertensive adult out-patients in a Kenyan referral hospital. Int J Pharm Pharm Sci. 2020 Nov 2;12(12):63-7. doi: 10.22159/ijpps.2020v12i12.38810.

7. Khodadadeh M. Exploring strategies for designing drug interac-tion clinical decision support systems: a qualitative study. Min-neapolis: Capella University; 2020.

8. Scheife RT, Hines LE, Boyce RD, Chung SP, Momper JD, Sommer CD. Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf. 2015 Feb;38(2):197-206. doi: 10.1007/s40264-014-0262-8, PMID 25556085, PMCID PMC4624322.

9. Sumayli AA, Daghriry AI, Sharahili MA, Alanazi AA, Alanazi AA, Alharbi IF. An in-depth examination of drug-drug interaction databases: enhancing patient safety through advanced predictive models and artificial intelligence techniques. Journal of Medical and Life Science. 2024 Dec 16;6(4):553-65. doi: 10.21608/jmals.2024.410645.

10. Munsaka M, Liu M, Xing Y, Yang H. Leveraging machine learning natural language processing and deep learning in drug safety and pharmacovigilance. In: Ghahramani Z, editor. Data science AI and machine learning in drug development. 1st ed. New York: Chapman & Hall/CRC; 2022 Oct 3. p. 193-229. doi: 10.1201/9781003150886-9.

11. Mutha RE, Bagul VS, Tade RS, Vinchurkar K. An overview of artificial intelligence (AI) in drug delivery and development. In: Vinchurkar K, editor. AI innovations in drug delivery and phar-maceutical sciences; advancing therapy through technology. Singapore: Bentham Science Publishers; 2024 Nov 18. p. 1-27. doi: 10.2174/9789815305753124010004.

12. Belagodu Sridhar S, Karattuthodi MS, Parakkal SA. Role of artifi-cial intelligence in clinical and hospital pharmacy. In: Bhupathy-raaj M, editor. Application of artificial intelligence in neurologi-cal disorders. Singapore: Springer Nature; 2024 Jul 1. p. 229-59. doi: 10.1007/978-981-97-2577-9_12.

13. TS, T CT, Marpaka S, KS. Aspects of utilization and limitations of artificial intelligence in drug safety. Asian J Pharm Clin Res. 2021 Aug;14(8):34-9. doi: 10.22159/ajpcr.2021.v14i8.41979.

14. Radha Krishnan RP, Hung EH, Ashford M, Edillo CE, Gardner C, Hatrick HB. Evaluating the capability of ChatGPT in predicting drug-drug interactions: real-world evidence using hospitalized patient data. Br J Clin Pharmacol. 2024 Dec;90(12):3361-6. doi: 10.1111/bcp.16275, PMID 39359001.

15. Shamim MA, Shamim MA, Arora P, Dwivedi P. Artificial intelli-gence and big data for pharmacovigilance and patient safety. Journal of Medicine Surgery and Public Health. 2024 Aug 1;3:100139. doi: 10.1016/j.glmedi.2024.100139.

16. Al Ashwal FY, Zawiah M, Gharaibeh L, Abu Farha R, Bitar AN. Evaluating the sensitivity specificity and accuracy of ChatGPT-3.5, ChatGPT-4, bing AI, and bard against conventional drug-drug interactions clinical tools. Drug Healthc Patient Saf. 2023 Dec 31;15:137-47. doi: 10.2147/DHPS.S425858, PMID 37750052.

17. Sulaiman DM, Shaba SS, Almufty HB, Sulaiman AM, Merza MA. Screening the drug-drug interactions between antimicrobials and other prescribed medications using google bard and Lexi-comp® Online™ database. Cureus. 2023 Sep 9;15(9):e44961. doi: 10.7759/cureus.44961, PMID 37692178, PMCID PMC10492649.

18. Aksoyalp ZS, Erdogan BR. Comparative evaluation of artificial intelligence and drug interaction tools: a perspective with the example of clopidogrel. Ankara Ecz Fak Derg. 2024;48(3):22. doi: 10.33483/jfpau.1460173.

19. Rambaran KA, Huynh HA, Zhang Z, Robles J. The Gap in elec-tronic drug information resources: a systematic review. Cureus. 2018 Jun 22;10(6):e2860. doi: 10.7759/cureus.2860, PMID 30148013, PMCID PMC6107040.

20. Shariff A, Belagodu Sridhar S, Abdullah Basha NF, Bin Taleth Alshemeil SS, Ahmed Aljallaf Alzaabi NA 4th. Assessing the con-sistency of drug-drug interaction-related information across various drug information resources. Cureus. 2021 Mar 8;13(3):e13766. doi: 10.7759/cureus.13766, PMID 33842142, PMCID PMC8025801.

21. Shareef J, Sridhar SB, Bhupathyraaj M, Shariff A, Thomas S, Salim Karattuthodi M. Assessment of the scope completeness and consistency of various drug information resources related to COVID-19 medications in pregnancy and lactation. BMC Preg-nancy Childbirth. 2023 Apr 27;23(1):296. doi: 10.1186/s12884-023-05609-2, PMID 37106456, PMCID PMC10134615.

22. Liu X, Hatton RC, Zhu Y, Hincapie Castillo JM, Bussing R, Barnicoat M. Consistency of psychotropic drug-drug interactions listed in drug monographs. J Am Pharm Assoc. 2017 Aug 23;57(6):698-703.e2. doi: 10.1016/j.japh.2017.07.008, PMID 28844584.

23. Marcath LA, Xi J, Hoylman EK, Kidwell KM, Kraft SL, Hertz DL. Comparison of nine tools for screening drug-drug interactions of oral oncolytics. J Oncol Pract. 2018 Jun;14(6):e368-74. doi: 10.1200/JOP.18.00086, PMID 29787332, PMCID PMC9797246.

24. Abhisek PA, Pradhan SS. Possible drug-drug interactions of hydroxychloroquine with concomitant medications in prophylaxis and treatment of COVID-19: multiple standard software-based assessment. J Clin Diagn Res. 2020 Dec 1;14(12):OC01-4. doi: 10.7860/JCDR/2020/45273.14392.

25. Beckett RD, Stump CD, Dyer MA. Evaluation of drug information resources for drug-ethanol and drug-tobacco interactions. J Med Libr Assoc. 2019 Jan;107(1):62-71. doi: 10.5195/jmla.2019.549, PMID 30598650, PMCID PMC6300238.

26. Patel RI, Beckett RD. Evaluation of resources for analyzing drug interactions. J Med Libr Assoc. 2016 Oct;104(4):290-5. doi: 10.3163/1536-5050.104.4.007, PMID 27822150, PMCID PMC5079490.

27. Alkhalid ZN, Birand N. Determination and comparison of poten-tial drug-drug interactions using three different databases in northern Cyprus community pharmacies. Niger J Clin Pract. 2022 Dec;25(12):2005-9. doi: 10.4103/njcp.njcp_448_22, PMID 36537458.

28. Pehlivanli A, Eren Sadioglu R, Aktar M, Eyupoglu S, Sengul S, Keven K. Potential drug-drug interactions of immunosuppres-sants in kidney transplant recipients: comparison of drug interaction resources. Int J Clin Pharm. 2022 Jun;44(3):651-62. doi: 10.1007/s11096-022-01385-9, PMID 35235113.

29. Bossaer JB, Eskens D, Gardner A. Sensitivity and specificity of drug interaction databases to detect interactions with recently approved oral antineoplastics. J Oncol Pharm Pract. 2022 Jan;28(1):82-6. doi: 10.1177/1078155220984244, PMID 33435823.

30. Juhi A, Pipil N, Santra S, Mondal S, Behera JK, Mondal H. The capability of ChatGPT in predicting and explaining common drug-drug interactions. Cureus. 2023 Mar 17;15(3):e36272. doi: 10.7759/cureus.36272, PMID 37073184, PMCID PMC10105894.

31. Askr H, Elgeldawi E, Aboul Ella H, Elshaier YA, Gomaa MM, Has-sanien AE. Deep learning in drug discovery: an integrative re-view and future challenges. Artif Intell Rev. 2023;56(7):5975-6037. doi: 10.1007/s10462-022-10306-1, PMID 36415536, PMCID PMC9669545.

32. Liu XH, Lu ZH, Wang T, Liu F. Large language models facilitating modern molecular biology and novel drug development. Front Pharmacol. 2024 Dec 24;15:1458739. doi: 10.3389/fphar.2024.1458739, PMID 39776586, PMCID PMC11703923.

33. Naik D, Naik I, Naik N. Imperfectly perfect AI chatbots: limita-tions of generative AI, large language models and large multi-modal models. In: Naik N, Jenkins P, Prajapat S, Grace P, editors. Contributions presented at the international conference on computing communication cybersecurity and AI, July 3-4, 2024, London, UK. Cham: Springer Nature Switzerland; 2024. p. 43-66. doi: 10.1007/978-3-031-74443-3_3.

34. Gill J, Moullet M, Martinsson A, Miljkovic F, Williamson B, Arends RH. Evaluating the performance of machine learning regression models for pharmacokinetic drug-drug interactions. CPT Pharmacometrics Syst Pharmacol. 2023 Jan;12(1):122-34. doi: 10.1002/psp4.12884, PMID 36382697, PMCID PMC9835131.

35. Xiong G, Yang Z, Yi J, Wang N, Wang L, Zhu H. DDInter: an online drug-drug interaction database towards improving clinical decision making and patient safety. Nucleic Acids Res. 2022 Jan 7;50(D1):D1200-7. doi: 10.1093/nar/gkab880, PMID 34634800, PMCID PMC8728114.