The Effectiveness of Local Fine-Tuned LLMs: Assessment of the Japanese National Examination for Pharmacists

Research Square (Research Square), Journal Year: 2025, Volume and Issue: unknown

Published: April 15, 2025

Large Language Models (LLMs) offer great potential for applications in healthcare and pharmaceutical fields. While cloud-based implementations are commonly used, they present challenges related to privacy cost. This study examined the performance of locally executable LLMs on Japanese National Examination Pharmacists (JNEP). Additionally, we explore feasibility creating specialized pharmacy models through fine-tuning with Low-Rank Adaptation (LoRA). Text-based questions from 97th 109th JNEP were utilized, comprising 2,421 training 165 testing. Four distinct evaluated, including Microsoft phi-4 DeepSeek R1 Distill Qwen series. Baseline was initially assessed, followed by using LoRA dataset. Model evaluated based accuracy scores achieved test In baseline evaluation against JNEP, ranged 55.15–76.36%. Notably, CyberAgent 32B passing threshold (approximately 61%). Following fine-tuning, exhibited a increase 60.61–66.06%. showed that capable handling knowledge tasks comparable those national pharmacist examination. Moreover, found techniques like can significantly enhance model performance, demonstrating robust AI specifically designed pharmacological applications. These findings contribute understanding implementing secure high-performing solutions tailored use.

Language: Английский

Integrating Large Language Models into Space Radiology: Opportunities and Challenges

Wilderness and Environmental Medicine, Journal Year: 2025, Volume and Issue: unknown

Published: April 30, 2025

Language: Английский