Enhancing Genetic Improvement Mutations Using Large Language Models DOI Creative Commons
Alexander E. I. Brownlee, James P. Callan, Karine Even-Mendoza

et al.

arXiv (Cornell University), Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs mutation operators for GI improve search process. We expand Gin Java toolkit call OpenAI's API generate edits JCodec tool. randomly sample space using 5 different edit types. find that number patches passing unit tests up 75% higher with LLM-based than standard Insert edits. Further, observe found are generally less diverse compared ran local runtime improvements. Although many improving by LLM-enhanced GI, best patch was GI.

Language: Английский

Acting Humanly: Identification and Analysis of Logical Reasoning Biases Exhibited by ChatGPT versus Undergraduate Students DOI
Ana Gabriela Caldas Oliveira, Francisco Leonardo Bezerra Martins, Davi Romero de Vasconcelos

et al.

Published: Nov. 17, 2024

Definitions of Artificial Intelligence (AI) include characterizing algorithms as those that: thinking humanly, rationally, acting humanly and rationally. On the one hand, Logic, a formal framework, allows for creation capable rationally by expressing real world situations in language that enables valid rigorous reasoning. other Large Language Models, such ChatGPT, represent especially tasks involving understanding generating natural text. However, these models can exhibit logical reasoning biases, which are tendencies impair ability to reason logically. This article aims identify analyze biases exhibited ChatGPT comparison Information Technology Undergraduate Students, beginners Logic course.

Language: Английский

Citations

0
Потенциал генеративного искусственного интеллекта для решения профессиональных задач DOI Creative Commons

Yaroslav Kouzminov,

Ekaterina Kruchinskaia

Foresight-Russia, Journal Year: 2024, Volume and Issue: 18(4), P. 67 - 76

Published: Dec. 9, 2024

Востребованность генеративного искусственного интеллекта (GenAI) стремительно растет ввиду способности быстро обрабатывать масштабные объемы данных, компилировать их и транслировать «общее мнение». Однако дисбаланс между «компетенциями» GenAI препятствует расширению использования этого инструмента для решения сложных профессиональных задач. ИИ работает как гигантский накопитель средство воспроизводства знаний, однако не способен интерпретировать находить правильное применение в зависимости от контекста. Сохраняется критическая вероятность ошибки при генерации ответов даже на самые простые вопросы. В статье оценивается степень значимости ограничений, присущих GenAI. Тестирование лежащих его основе языковых моделей, включая новейшие версии — GPT-4o1 GigaChat MAX, проводилось с помощью авторского набора вопросов, основанного таксономии Блума. Установлено, что получения правильного ответа практически зависит количества параметров настройки, сложности таксономии, а наличии множественного выбора снижается. Полученные результаты подтверждают предположение о невозможности применения современных инструментов целях. Предлагаются опции, способные внести значимый вклад достижение минимум квазипрофессионального уровня.

Language: Русский

Citations

0
Is ChatGPT Humanly Irrational? DOI Creative Commons
Ding Ma, Tongda Zhang, Michael A. Saunders

et al.

Research Square (Research Square), Journal Year: 2023, Volume and Issue: unknown

Published: Sept. 22, 2023

Abstract We delve into the fascinating crossroads of artificial intelligence (AI) and cognitive science, spotlighting OpenAI advanced language model, ChatGPT. Renowned for generating human-like text, ChatGPT has been widely used in various applications. However, its ability to replicate human processes, particularly decision-making behavior, remains largely unexplored untapped. evaluate ChatGPT's patterns show that they strikingly mirror those subjects, even traditionally termed ''irrational'' under standard economic theory. This finding challenges prevailing assumption AI systems operate solely on rational computations. It suggests that, despite algorithmic nature, can reflect biases when simulating roles, thus adding a new dimension our understanding behaviour. Our result places models like broader context indicating their potential mimic not just but also processes. From perspective, findings underscore capacity behavioral research stimulate necessary dialogue design, transparency, ethical implications. study bridges machine intelligence, highlighting enhance processes agents.

Language: Английский

Citations

0
Enhancing Genetic Improvement Mutations Using Large Language Models DOI Creative Commons
Alexander E. I. Brownlee, James P. Callan, Karine Even-Mendoza

et al.

arXiv (Cornell University), Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs mutation operators for GI improve search process. We expand Gin Java toolkit call OpenAI's API generate edits JCodec tool. randomly sample space using 5 different edit types. find that number patches passing unit tests up 75% higher with LLM-based than standard Insert edits. Further, observe found are generally less diverse compared ran local runtime improvements. Although many improving by LLM-enhanced GI, best patch was GI.

Language: Английский

Citations

0