As set out in the 2024 BMA report Principles for artificial intelligence (AI) and its application in healthcare, AI can support medical research and learning for doctors. The ability to use natural language models (or large language models, LLMs) can be particularly appealing due to the speed at which these tools work. However, generative LLM often referred to as AI, particularly those offered free of charge or open access, (such as Gemini, Claude and ChatGPT) pose significant risks when utilised in medical research and learning. Many AI tools offer little to no transparency in the strategy employed or the content explored for the search[1] which can enforce and embed bias[2], already present in the AI or in the data available in the model, onto the results produced. This lack of transparency means that users are not able to apply their professional judgement.

LLMs learn from the content entered by users which means you would be allowing your intellectual property and potentially copyrighted content to be ingested into the AI model where terms and conditions mean that they will be able to use this in the future over which you have no control. The article Doctors at risk of ‘intellectual theft’ by robots explores this concept.

 When used for developing an evidence-based literature search there are reported instances of AI providing cited results for the user, where the citations are hallucinated, follow a recognised structure and sound authoritative but are entirely fabricated. As well as the issues of using LLM to support literature reviews and searching, AI models have been found to engage in unethical actions when applied to support research more widely such as invading privacy and deceiving human research participants[3].

In addition to the issues facing the use of AI in research there are significant holes in the data accessible by these open access models as many publishers prevent their propriety data from being uploaded. Users must also remember that in accessing published material either through purchase, subscription or via a library they are agreeing to copyright restrictions which means that full text content cannot be ingested into AI models, an activity which might be appealing to provide summaries.

Reproducibility, validation, and adversarial-review are research-integrity competencies, the use of LLMs can impact all these but this does not mean that you can make use of tools to support with elements of your research especially as many publishers have refined their “no-AI-co-author” policies to disclosure templates required at the point of submission[4].

Key risk of Open Access AI

• No transparency in results
• Inherent bias in the data creates bias in results
• IP or copyright infringement by the tools
• Hallucinated results
• Gaps in data

Elsevier offers an accredited self-paced Gen AI Academy for Health course free.

Many publications are exploring the use of AI in medicine, BMA Library has curated a collection of titles covering a wide range of AI topics all produced by trusted publishers, you can access the collection here.

[1] Living guidelines on the responsible use of generative AI in researchERA Forum Stakeholders’ document Third version, May 2026 – https://www.google.com/goto?url=CAESwQEB6zswFb-ZfkxAHQwgj7B7ZgvDJNIT_Y3xNzFuctzU0iAnwekFWX49rCenieqa0ZddF7nUgdbBr8KnfdLyuEI2B9V7AdJao2V1cM_r_CJo2MetRuZi0EkwUCmjDjma6sIfcqthL4E5vp953vLqtHn9JYHSL6kTEI-ObvUWsP0M0-zrAI6AUkpuB2zdPDHCmAuw1HqqtLdMRhQu6iaKKzrhNUgaOW1LLVSgrlPAmMOlsWwV_4uhq59mLljl7pMz4EJh  

[2] Schwendicke F, Sidhu SK, Ferracane JL, Tichy A, Jakubovics NS. Generative AI: Opportunities, Risks, and Responsibilities for Oral Sciences. J Dent Res. 2025 Dec;104(13):1429-1431. doi: 10.1177/00220345251356408. Epub 2025 Oct 15. PMID: 41091144; PMCID: PMC12578959

[3] Hosseini, Mohammad & Murad, Maya & Resnik, David. (2025). Benefits and Risks of Using AI Agents in Research. 10.31235/osf.io/x8sgj_v1.

[4]  Zyphur, M. J. (2026). Responsible AI in Academic Research: A Competency Framework for Research Training. Instats Policy Series. github.com/mzyphur/responsible-ai-in-research-training. ORCID: 0000-0003-3237-7892. DOI: 10.61700/t31oy23grr.