Software Testing With Large Language Models: Survey, Landscape, and Vision

IEEE Transactions on Software Engineering, Journal Year: 2024, Volume and Issue: 50(4), P. 911 - 936

Published: Feb. 20, 2024

Pre-trained large language models (LLMs) have recently emerged as a breakthrough technology in natural processing and artificial intelligence, with the ability to handle large-scale datasets exhibit remarkable performance across wide range of tasks. Meanwhile, software testing is crucial undertaking that serves cornerstone for ensuring quality reliability products. As scope complexity systems continue grow, need more effective techniques becomes increasingly urgent, making it an area ripe innovative approaches such use LLMs. This paper provides comprehensive review utilization LLMs testing. It analyzes 102 relevant studies used testing, from both perspectives. The presents detailed discussion tasks which are commonly used, among test case preparation program repair most representative. also LLMs, types prompt engineering employed, well accompanied these summarizes key challenges potential opportunities this direction. work can serve roadmap future research area, highlighting avenues exploration, identifying gaps our current understanding

Language: Английский

---

When LLMs meet cybersecurity: a systematic literature review

Cybersecurity, Journal Year: 2025, Volume and Issue: 8(1)

Published: Feb. 5, 2025

Abstract The rapid development of large language models (LLMs) has opened new avenues across various fields, including cybersecurity, which faces an evolving threat landscape and demand for innovative technologies. Despite initial explorations into the application LLMs in there is a lack comprehensive overview this research area. This paper addresses gap by providing systematic literature review, covering analysis over 300 works, encompassing 25 more than 10 downstream scenarios. Our three key questions: construction cybersecurity-oriented LLMs, to cybersecurity tasks, challenges further study aims shed light on extensive potential enhancing practices serve as valuable resource applying field. We also maintain regularly update list practical guides at https://github.com/tmylla/Awesome-LLM4Cybersecurity .

Language: Английский

---

A Review on Edge Large Language Models: Design, Execution, and Applications

ACM Computing Surveys, Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 24, 2025

Large language models (LLMs) have revolutionized natural processing with their exceptional understanding, synthesizing, and reasoning capabilities. However, deploying LLMs on resource-constrained edge devices presents significant challenges due to computational limitations, memory constraints, hardware heterogeneity. This survey provides a comprehensive overview of recent advancements in LLMs, covering the entire lifecycle — from resource-efficient model design pre-deployment strategies runtime inference optimizations. It also explores on-device applications across various domains. By synthesizing state-of-the-art techniques identifying future research directions, this bridges gap between immense potential constraints computing.

Language: Английский

---

Towards an understanding of large language models in software engineering tasks

Empirical Software Engineering, Journal Year: 2024, Volume and Issue: 30(2)

Published: Dec. 26, 2024

Language: Английский

---

Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmap

ACM Transactions on Autonomous and Adaptive Systems, Journal Year: 2024, Volume and Issue: unknown

Published: Aug. 20, 2024

Self-adaptive systems (SASs) are designed to handle changes and uncertainties through a feedback loop with four core functionalities: monitoring, analyzing, planning, execution. Recently, generative artificial intelligence (GenAI), especially the area of large language models, has shown impressive performance in data comprehension logical reasoning. These capabilities highly aligned functionalities required SASs, suggesting strong potential employ GenAI enhance SASs. However, specific benefits challenges employing SASs remain unclear. Yet, providing comprehensive understanding these is complex due several reasons: limited publications SAS field, technological application diversity within rapid evolution technologies. To that end, this paper aims provide researchers practitioners snapshot outlines GenAI’s SAS. Specifically, we gather, filter, analyze literature from distinct research fields organize them into two main categories benefits: (i) enhancements autonomy centered around functions MAPE-K loop, (ii) improvements interaction between humans human-on-the-loop settings. From our study, outline roadmap highlights integrating The starts outlining key need be tackled exploit for applying field concludes practical reflection, elaborating on current shortcomings proposing possible mitigation strategies. 1

Language: Английский

---

Multitask-based Evaluation of Open-Source LLM on Software Vulnerability

IEEE Transactions on Software Engineering, Journal Year: 2024, Volume and Issue: 50(11), P. 3071 - 3087

Published: Oct. 7, 2024

Language: Английский

---

CodeDoctor: multi-category code review comment generation

Automated Software Engineering, Journal Year: 2025, Volume and Issue: 32(1)

Published: Feb. 27, 2025

Language: Английский

---

Advancing software security: DCodeBERT for automatic vulnerability detection and repair

Journal of Industrial Information Integration, Journal Year: 2025, Volume and Issue: unknown, P. 100834 - 100834

Published: March 1, 2025

Language: Английский

---

PAFL: Enhancing Fault Localizers by Leveraging Project-Specific Fault Patterns

Proceedings of the ACM on Programming Languages, Journal Year: 2025, Volume and Issue: 9(OOPSLA1), P. 1378 - 1405

Published: April 9, 2025

We present PAFL, a new technique for enhancing existing fault localization methods by leveraging project-specific patterns. observed that each software project has its own challenges and suffers from recurring patterns associated with those challenges. However, techniques use universal strategy without considering repetitive faults. To address this limitation, our technique, called project-aware (PAFL), enables localizers to leverage Given buggy version of baseline localizer, PAFL first mines the past versions project. Then, it uses mined update suspiciousness scores statements computed localizer. end, we two novel ideas. First, design domain-specific pattern-description language represent various An instance, crossword, in describes pattern how affects statements. Second, develop an algorithm synthesizes crosswords (i.e., patterns) Evaluation using seven 12 real-world C/C++ Python projects demonstrates effectively, robustly, efficiently improves performance techniques.

Language: Английский

---

FLAG: F inding L ine A nomalies (in RTL code) with G enerative AI

ACM Transactions on Design Automation of Electronic Systems, Journal Year: 2025, Volume and Issue: unknown

Published: May 20, 2025

Bug detection in Hardware Design Languages (HDLs) is an important problem the System-on-Chip (SoC) development cycle. It crucial to find defects at earliest stage possible. While most fault localization requires use of ‘tests’ (e.g. test benches, fuzzing and assertions) a simulation or emulation framework, advent Large Language Models (LLMs) provides opportunity for test-free approach. This paper proposes such tool, called FLAG, which can identify functional security Register Transfer Level (RTL) code without synthesis simulation. FLAG combines syntactic generative AI techniques implement RTL code. takes design as input outputs set line(s) that likely contain defects. targets elements bugs through static analysis means then implements token-level line-level obtain differences original generated by LLM line buggy not. The approach evaluates each token (one time) level entire LLM. We evaluate our on corpus synthetic real-world bugs, both related issues, Verilog SystemVerilog. Using analysis, 38 out 120 using 32 81 top-5 bug locations identified tests.

Language: Английский

---