Fine-tuning and prompt engineering for large language models-based code review automation

Information and Software Technology, Journal Year: 2024, Volume and Issue: 175, P. 107523 - 107523

Published: July 11, 2024

Language: Английский

---

Clover: Closed-Loop Verifiable Code Generation

Lecture notes in computer science, Journal Year: 2024, Volume and Issue: unknown, P. 134 - 155

Published: Jan. 1, 2024

Language: Английский

---

Investigating large language models capabilities for automatic code repair in Python

Cluster Computing, Journal Year: 2024, Volume and Issue: 27(8), P. 10717 - 10731

Published: May 9, 2024

Language: Английский

---

Automatic Generation of Loop Invariants in Dafny with Large Language Models

Lecture notes in computer science, Journal Year: 2025, Volume and Issue: unknown, P. 138 - 154

Published: Jan. 1, 2025

Language: Английский

---

Challenges and Paths Towards AI for Software Engineering

Published: April 4, 2025

AI for software engineering has made remarkable progress recently, becoming a notable success within generative AI. Despite this, there are still many challenges that need to be addressed before automated reaches its full potential. It should possible reach high levels of automation where humans can focus on the critical decisions what build and how balance difficult tradeoffs while most routine development effort is away. Reaching this level will require substantial research efforts across academia industry. In paper, we aim discuss towards in threefold manner. First, provide structured taxonomy concrete tasks engineering, emphasizing other beyond code generation completion. Second, outline several key bottlenecks limit current approaches. Finally, an opinionated list promising directions toward making these bottlenecks, hoping inspire future rapidly maturing field.

Language: Английский

---

Metamorph: Synthesizing Large Objects from Dafny Specifications

Proceedings of the ACM on Programming Languages, Journal Year: 2025, Volume and Issue: 9(OOPSLA1), P. 759 - 785

Published: April 9, 2025

Program synthesis aims to produce code that adheres user-provided specifications. In this work, we focus on synthesizing sequences of calls formally specified APIs generate objects satisfy certain properties. This problem is particularly relevant in automated test generation, where a engine may need an object with specific properties trigger given execution path. Constructing instances complex data structures require dozens method calls, but reasoning about consecutive computationally expensive, and existing work typically limits the number solution. paper, such long Dafny programming language. To end, introduce Metamorph, tool uses counterexamples returned by verifier reason effects one at time, limiting complexity solver queries. We also aim limit overall SMT queries comparing using two distance metrics develop for guiding process. particular, novel piecewise metric, which puts provably correct lower bound solution allows us frame as weighted A* search. When computing distance, view states conjunctions atomic constraints, identify constraints each call can satisfy, combine information integer programming. evaluate Metamorph’s ability large six benchmarks defining key structures: linked lists, queues, arrays, binary trees, graphs. Metamorph successfully construct programs up 57 per instance compares favorably alternative baseline approach. Additionally, integrate DTest, Dafny’s generation toolkit, show synthesize inputs parts AWS Cryptographic Material Providers Library DTest alone not able cover. Finally, use executable bytecode simple virtual machine, demonstrating techniques described here are more broadly applicable context specification-guided synthesis.

Language: Английский

---

Laurel: Unblocking Automated Verification with Large Language Models

Proceedings of the ACM on Programming Languages, Journal Year: 2025, Volume and Issue: 9(OOPSLA1), P. 1519 - 1545

Published: April 9, 2025

Program verifiers such as Dafny automate proofs by outsourcing them to an SMT solver. This automation is not perfect, however, and the solver often requires hints in form of assertions , creating a burden for proof engineer. In this paper, we propose tool that alleviates automatically generating using large language models (LLMs). To improve success rate LLMs task, design two domain-specific prompting techniques. First, help LLM determine location missing assertion analyzing verifier’s error message inserting placeholder at location. Second, provide with example from same codebase, which select based on new similarity metric. We evaluate our techniques benchmark dataset complex lemmas extracted three real-world codebases. Our evaluation shows able generate over 56.6% required given only few attempts, making affordable unblocking program without human intervention.

Language: Английский

---

Requirements are All You Need: The Final Frontier for End-User Software Engineering

ACM Transactions on Software Engineering and Methodology, Journal Year: 2024, Volume and Issue: unknown

Published: Dec. 14, 2024

What if end users could own the software development life cycle from conception to deployment using only requirements expressed in language, images, video or audio? We explore this idea, building on capabilities that Generative Artificial Intelligence brings generation and maintenance techniques. How designing way better serve users? are implications of process for future end-user engineering cycle? discuss research needed bridge gap between where we today these imagined systems future.

Language: Английский

---