GPT (Generative Pre-Trained Transformer)— A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

IEEE Access, Год журнала: 2024, Номер 12, С. 54608 - 54649

Опубликована: Янв. 1, 2024

The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward development machines that can understand and communicate using manner closely resembles humans. GPT based on transformer architecture, deep neural network designed for processing tasks. Due to their impressive performance tasks ability effectively converse, have gained significant popularity among researchers industrial communities, making them one most widely used effective models related fields, motivated conduct this review. This review provides detailed overview GPT, including its working process, training procedures, enabling technologies, impact various applications. In review, we also explored potential challenges limitations GPT. Furthermore, discuss solutions future directions. Overall, paper aims provide comprehensive understanding applications, emerging challenges, solutions.

Язык: Английский

---

Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration

Healthcare, Год журнала: 2023, Номер 11(20), С. 2776 - 2776

Опубликована: Окт. 20, 2023

Generative artificial intelligence (AI) and large language models (LLMs), exemplified by ChatGPT, are promising for revolutionizing data information management in healthcare medicine. However, there is scant literature guiding their integration non-AI professionals. This study conducts a scoping review to address the critical need guidance on integrating generative AI LLMs into medical practices. It elucidates distinct mechanisms underpinning these technologies, such as Reinforcement Learning from Human Feedback (RLFH), including few-shot learning chain-of-thought reasoning, which differentiates them traditional, rule-based systems. requires an inclusive, collaborative co-design process that engages all pertinent stakeholders, clinicians consumers, achieve benefits. Although global research examining both opportunities challenges, ethical legal dimensions, offer advancements enhancing management, retrieval, decision-making processes. Continued innovation acquisition, model fine-tuning, prompt strategy development, evaluation, system implementation imperative realizing full potential of technologies. Organizations should proactively engage with technologies improve quality, safety, efficiency, adhering guidelines responsible application.

Язык: Английский

---

HuatuoGPT, Towards Taming Language Model to Be a Doctor

Опубликована: Янв. 1, 2023

Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Guiming Jianquan Li, Xiangbo Wu, Zhang Zhiyi, Qingying Xiao, Xiang Wan, Benyou Wang, Haizhou Li. Findings of the Association for Computational Linguistics: EMNLP 2023.

Язык: Английский

---

BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights

Journal of the American Medical Informatics Association, Год журнала: 2024, Номер 31(9), С. 1844 - 1855

Опубликована: Фев. 27, 2024

Abstract Objective In this study, we investigate the potential of large language models (LLMs) to complement biomedical knowledge graphs in training semantic for and clinical domains. Materials Methods Drawing on wealth Unified Medical Language System graph harnessing cutting-edge LLMs, propose a new state-of-the-art approach obtaining high-fidelity representations concepts sentences, consisting 3 steps: an improved contrastive learning phase, novel self-distillation weight averaging phase. Results Through rigorous evaluations diverse downstream tasks, demonstrate consistent substantial improvements over previous state art textual similarity (STS), concept representation (BCR), clinically named entity linking, across 15+ datasets. Besides our model English, also distill release multilingual compatible with 50+ languages finetuned 7 European languages. Discussion Many pipelines can benefit from latest models. Our enables range advancements learning, opening avenue bioinformatics researchers around world. As result, hope see BioLORD-2023 becoming precious tool future applications. Conclusion article, introduced BioLORD-2023, STS BCR designed domain.

Язык: Английский

---

Enhancing Accuracy in Large Language Models Through Dynamic Real-Time Information Injection

Опубликована: Дек. 26, 2023

This study presents a novel approach to enhance Large Language Models (LLMs) like Alpaca by dynamically integrating real-time information. method addresses the issue of content hallucination and data relevancy automatically collecting current from credible sources into model prompts. Experiments show significant improvement in accuracy decrease hallucination, with manageable increase response time. The research underscores potential integration making LLMs more accurate contextually relevant, setting foundation for future advancements dynamic processing AI.

Язык: Английский

---

Me-LLaMA: Foundation Large Language Models for Medical Applications

Research Square (Research Square), Год журнала: 2024, Номер unknown

Опубликована: Май 22, 2024

Recent advancements in large language models (LLMs) such as ChatGPT and LLaMA have hinted at their potential to revolutionize medical applications, yet application clinical settings often reveals limitations due a lack of specialized training on medical-specific data. In response this challenge, study introduces Me-LLaMA, novel LLM family that includes foundation – Me-LLaMA 13/70B, along with chat-enhanced versions 13/70B-chat, developed through continual pre-training instruction tuning LLaMA2 using datasets. Our methodology leverages comprehensive domain-specific data suite, including large-scale, dataset 129B tokens, an 214k samples, new evaluation benchmark (MIBE) across six critical tasks 12 extensive the MIBE shows achieve overall better performance than existing open-source LLMs zero-shot, few-shot supervised learning abilities. With task-specific tuning, outperform 7 out 8 datasets GPT-4 5 addition, we investigated catastrophic forgetting problem, our results show other mitigating issue. is one largest use both biomedical It exhibits superior general compared LLMs, rendering it attractive choice for AI applications. We release models, datasets, scripts at: https://github.com/BIDS-Xu-Lab/Me-LLaMA.

Язык: Английский

---

Global insights and the impact of generative AI-ChatGPT on multidisciplinary: a systematic review and bibliometric analysis

Connection Science, Год журнала: 2024, Номер 36(1)

Опубликована: Май 16, 2024

In 2022, OpenAI's unveiling of generative AI Large Language Models (LLMs)- ChatGPT, heralded a significant leap forward in human-machine interaction through cutting-edge technologies. With its surging popularity, scholars across various fields have begun to delve into the myriad applications ChatGPT. While existing literature reviews on LLMs like ChatGPT are available, there is notable absence systematic (SLRs) and bibliometric analyses assessing research's multidisciplinary geographical breadth. This study aims bridge this gap by synthesising evaluating how has been integrated diverse research areas, focussing scope distribution studies. Through review scholarly articles, we chart global utilisation scientific domains, exploring contribution advancing paradigms adoption trends among different disciplines. Our findings reveal widespread endorsement multiple fields, with implementations healthcare (38.6%), computer science/IT (18.6%), education/research (17.3%). Moreover, our demographic analysis underscores ChatGPT's reach accessibility, indicating participation from 80 unique countries ChatGPT-related research, most frequent keyword occurrence, USA (719), China (181), India (157) leading contributions. Additionally, highlights roles institutions such as King Saud University, All Institute Medical Sciences, Taipei University pioneering dataset. not only sheds light vast opportunities challenges posed pursuits but also acts pivotal resource for future inquiries. It emphasises that (LLM) role revolutionising every field. The insights provided paper particularly valuable academics, researchers, practitioners disciplines, well policymakers looking grasp extensive impact technologies community.

Язык: Английский

---

Parameter-efficient fine-tuning large language model approach for hospital discharge paper summarization

Applied Soft Computing, Год журнала: 2024, Номер 157, С. 111531 - 111531

Опубликована: Март 24, 2024

Язык: Английский

---

Fairness in Large Language Models: A Taxonomic Survey

ACM SIGKDD Explorations Newsletter, Год журнала: 2024, Номер 26(1), С. 34 - 48

Опубликована: Июль 24, 2024

Large Language Models (LLMs) have demonstrated remarkable success across various domains. However, despite their promising performance in numerous real-world applications, most of these algorithms lack fairness considerations. Consequently, they may lead to discriminatory outcomes against certain communities, particularly marginalized populations, prompting extensive study fair LLMs. On the other hand, LLMs, contrast traditional machine learning, entails exclusive backgrounds, taxonomies, and fulfillment techniques. To this end, survey presents a comprehensive overview recent advances existing literature concerning Specifically, brief introduction LLMs is provided, followed by an analysis factors contributing bias Additionally, concept discussed categorically, summarizing metrics for evaluating promoting fairness. Furthermore, resources including toolkits datasets, are summarized. Finally, research challenges open questions discussed.

Язык: Английский

---

Human-Comparable Sensitivity of Large Language Models in Identifying Eligible Studies Through Title and Abstract Screening: 3-Layer Strategy Using GPT-3.5 and GPT-4 for Systematic Reviews

Journal of Medical Internet Research, Год журнала: 2024, Номер 26, С. e52758 - e52758

Опубликована: Авг. 16, 2024

Background The screening process for systematic reviews is resource-intensive. Although previous machine learning solutions have reported reductions in workload, they risked excluding relevant papers. Objective We evaluated the performance of a 3-layer method using GPT-3.5 and GPT-4 to streamline title abstract-screening reviews. Our goal develop that maximizes sensitivity identifying records. Methods conducted screenings on 2 our related treatment bipolar disorder, with 1381 records from first review 3146 second. Screenings were (gpt-3.5-turbo-0125) (gpt-4-0125-preview) across three layers: (1) research design, (2) target patients, (3) interventions controls. was prompts tailored each study. During this process, information extraction according study’s inclusion criteria optimization carried out GPT-4–based flow without manual adjustments. Records at layer, those meeting all layers subsequently judged as included. Results On both able about 110 per minute, total time required second studies approximately 1 hour hours, respectively. In study, sensitivities/specificities 0.900/0.709 0.806/0.996, Both by 6 used meta-analysis 0.958/0.116 0.875/0.855, sensitivities align human evaluators: 0.867-1.000 study 0.776-0.979 9 After accounting justifiably excluded GPT-4, 0.962/0.996 0.943/0.855 Further investigation indicated cases incorrectly due lack domain knowledge, while misinterpretations criteria. Conclusions demonstrated acceptable level specificity supports its practical application screenings. Future should aim generalize approach explore effectiveness diverse settings, medical nonmedical, fully establish use operational feasibility.

Язык: Английский

---