ChatGPT for Education and Research: Opportunities, Threats, and Strategies

Applied Sciences, Journal Year: 2023, Volume and Issue: 13(9), P. 5783 - 5783

Published: May 8, 2023

In recent years, the rise of advanced artificial intelligence technologies has had a profound impact on many fields, including education and research. One such technology is ChatGPT, powerful large language model developed by OpenAI. This offers exciting opportunities for students educators, personalized feedback, increased accessibility, interactive conversations, lesson preparation, evaluation, new ways to teach complex concepts. However, ChatGPT poses different threats traditional research system, possibility cheating online exams, human-like text generation, diminished critical thinking skills, difficulties in evaluating information generated ChatGPT. study explores potential that overall from perspective educators. Furthermore, programming learning, we explore how helps improve their skills. To demonstrate this, conducted coding-related experiments with code generation problem descriptions, pseudocode algorithms texts, correction. The codes are validated an judge system evaluate accuracy. addition, several surveys teachers find out supports learning teaching. Finally, present survey results analysis.

Language: Английский

---

An Analysis of the Automatic Bug Fixing Performance of ChatGPT

Published: May 1, 2023

To support software developers in finding and fixing bugs, several automated program repair techniques have been introduced. Given a test suite, standard methods usually either synthesize repair, or navigate search space of edits to find test-suite passing variants. Recent are based on deep learning approaches. One these novel methods, which is not primarily intended for but still suitable it, ChatGPT. The bug performance ChatGPT, however, so far unclear. Therefore, this paper we evaluate ChatGPT the benchmark set, QuixBugs, compare with results other approaches reported literature. We that ChatGPT's competitive common CoCoNut Codex notably better than In contrast previous approaches, offers dialogue system through further information, e.g., expected output certain input an observed error message, can be entered. By providing such hints its success rate increased, 31 out 40 outperforming state-of-the-art.

Language: Английский

---

ChatGPT for Education and Research: Opportunities, Threats, and Strategies

Published: March 28, 2023

In recent years, the rise of advanced artificial intelligence technologies has had a profound impact on many fields, including education and research. One such technology is ChatGPT, powerful large language model developed by OpenAI. This offers exciting opportunities for students educators, personalized feedback, increased accessibility, interactive conversations, lesson preparation, evaluation, new ways to teach complex concepts. However, ChatGPT poses different threats traditional research system, possibility cheating online exams, human-like text generation, diminished critical thinking skills, difficulties in evaluating information generated ChatGPT. study explores potential that overall from perspective educators. Furthermore, programming learning, we explore how helps improve their skills. To demonstrate this, conducted coding-related experiments with code generation problem descriptions, pseudocode algorithms texts, correction. We also verified codes an judge system evaluate accuracy.

Language: Английский

---

Data cleaning and machine learning: a systematic literature review

Automated Software Engineering, Journal Year: 2024, Volume and Issue: 31(2)

Published: June 11, 2024

Language: Английский

---

An Empirical Study on Fine-Tuning Large Language Models of Code for Automated Program Repair

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), Journal Year: 2023, Volume and Issue: unknown, P. 1162 - 1174

Published: Sept. 11, 2023

The advent of large language models (LLMs) has opened up new opportunities for automated program repair (APR). In particular, some recent studies have explored how to leverage code (LLMCs) tasks and show promising results. However, most them adopt the zero/few-shot learning paradigm APR, which directly use LLMCs generate possibly correct given its surrounding context. Though effective, capabilities based on fine-tuning yet be extensively explored. Also, it remains unknown whether potential more complicated bugs (e.g., multi-hunk bugs). To fill gap, in this work, we conduct a comprehensive study capability paradigm. We select 5 popular with representative pre-training architectures, including CodeBERT, GraphCode-BERT, PLBART, CodeT5, UniX coder. consider 3 typical scenarios (i.e., bugs, vulnerabilities, errors) involving programming languages Java, $\mathrm{C}/\mathrm{C}++$ , JavaScript). Notably, take both single-hunk bugs/vulnerabilities into account. then fine-tune widely-used datasets compare existing state-of-the-art APR tools. also investigate impact different design choices, include abstractions, representations, model evaluation metrics. Our experimental results that can significantly outperform previous Through in-depth analysis, provide insights choosing appropriate strategies guide better performance. Lastly, reveal several limitations make suggestions future research LLMC-based APR.

Language: Английский

---

Enhanced automated code vulnerability repair using large language models

Engineering Applications of Artificial Intelligence, Journal Year: 2024, Volume and Issue: 138, P. 109291 - 109291

Published: Sept. 10, 2024

Language: Английский

---

Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road Ahead

ACM Transactions on Software Engineering and Methodology, Journal Year: 2024, Volume and Issue: unknown

Published: Dec. 18, 2024

The significant advancements in Large Language Models (LLMs) have resulted their widespread adoption across various tasks within Software Engineering (SE), including vulnerability detection and repair. Numerous studies investigated the application of LLMs to enhance repair tasks. Despite increasing research interest, there is currently no existing survey that focuses on utilization for In this paper, we aim bridge gap by offering a systematic literature review approaches aimed at improving through LLMs. encompasses work from leading SE, AI, Security conferences journals, encompassing 43 papers published 25 distinct venues, along with 15 high-quality preprint papers, bringing total 58 papers. By answering three key questions, (1) summarize employed relevant literature, (2) categorize LLM adaptation techniques detection, (3) classify Based our findings, identified series limitations studies. Additionally, outlined roadmap highlighting potential opportunities believe are pertinent crucial future endeavors.

Language: Английский

---

Gamma: Revisiting Template-Based Automated Program Repair Via Mask Prediction

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), Journal Year: 2023, Volume and Issue: unknown, P. 535 - 547

Published: Sept. 11, 2023

Automated program repair (APR) aims to fix software bugs without manual debugging efforts and plays a crucial role in development maintenance. Template-based APR has been widely investigated shown promising results. However, it is challenging for template-based select the appropriate donor code, which an important ingredient generating candidate patches. Inappropriate code may cause plausible but incorrect patch generation even with correct patterns, limiting performance. In this paper, we aim revisit APR, propose Gamma, directly leverage large pre-trained language models generation. Our main insight that instead of retrieving local buggy file, can predict tokens based on context snippets patterns by cloze task. Specifically, (1) Gamma revises variety templates from state-of-the-art techniques (i.e., TBar) transforms them into mask patterns. (2) adopts model masked as fill-in-the-blank Although our idea general be built various existing models, have implemented practical tool recent UniXcoder model. The experimental results demonstrate correctly repairs 82 Defects4J-v1.2, achieves 20.59% (14 bugs) 26.15% (17 improvement over previous approach TBar learning-based one Recoder. Furthermore, 45 22 additional Defects4J-v2.0 QuixBugs, indicating generalizability addressing dataset overfitting issue. We also prove adopting other provide substantial advancement, e.g., CodeBERT-based ChatGPT-based able 80 67 scalability Gamma. Overall, study highlights future generate patches top practice.

Language: Английский

---

Pre-Trained Model-Based Automated Software Vulnerability Repair: How Far are We?

IEEE Transactions on Dependable and Secure Computing, Journal Year: 2023, Volume and Issue: 21(4), P. 2507 - 2525

Published: Aug. 28, 2023

Various approaches are proposed to help under-resourced security researchers detect and analyze software vulnerabilities. It is still incredibly time-consuming labor-intensive for fix such reported vulnerabilities due the increasing size complexity of modern systems. The time lag between reporting fixing a vulnerability causes systems suffer from significant exposure possible attacks. Very recently, some techniques propose apply pretrained models have proved their success in improving repair accuracy. However, effectiveness existing pre-trained has not been systematically compared little known about advantages disadvantages. To bridge this gap, we perform first extensive study on applying various automated repair. experimental results two datasets show that all studied consistently outperform state-ofthe- art technique VRepair with prediction accuracy 32.94 $\sim$ 44.96%. We also investigate impact three major phases (i.e., data pre-processing, model training inference) workflow. Inspired by findings, construct simplistic approach adopts transfer learning bug fixing. Surprisingly, can further improve 9.40% average. Besides, provide additional discussion different aspects (e.g., code representation preliminary ChatGPT) illustrate capacity limitation model-based techniques. Finally, pinpoint practical guidelines improvement fine-tuning) advanced near future. Our highlights promising future adopting patch real-world reduce manual debugging effort experts practice.

Language: Английский

---

Investigating large language models capabilities for automatic code repair in Python

Cluster Computing, Journal Year: 2024, Volume and Issue: 27(8), P. 10717 - 10731

Published: May 9, 2024

Language: Английский

---