Test-Driven Development Using LLM: A look into LLMs writing tests in a test-driven development workflow
2024 (English)Independent thesis Basic level (degree of Bachelor), 180 HE credits
Student thesis
Abstract [en]
This research aims to explore different Large Language Models (LLM) and drawconclusions as to how well they can generate edge cases and respective unit tests in aTest-Driven Development (TDD) workflow. The experiment is concluded with LLMsthat are of the type Generative Pre-trained Transformers (GPT). The different LLMsthat are used for this research are ChatGPT 4 (GPT-4), ChatGPT 3.5 (GPT3.5) andGithub Copilot (Codex/GPT-4). The experiment consists of two phases, one where theprompt goes through prompt engineering with different methods such as few-shot andChain of Thought (COT). The second phase is where the prompt is sent to thedifferent LLMs to generate edge cases and unit tests. The prompt and questions thatare sent and asked to the LLM are taken from Advent of Code (AoC) which is aprogramming event every year. The LLMs performed differently depending on thetype of question from AoC. ChatGPT 3.5 performed better during text heavyquestions, ChatGPT 4 had an average performance throughout all the questions andGithub Copilot performed better towards the end where the questions became morecomplex and more programmatically focused.
Place, publisher, year, edition, pages
2024. , p. 51
Keywords [en]
Artificial Intelligence, AI, Test-Driven Development, TDD, prompt engineering
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:hj:diva-65413OAI: oai:DiVA.org:hj-65413DiVA, id: diva2:1878232
External cooperation
Combitech AB
Subject / course
JTH, Computer Engineering
Supervisors
Examiners
2024-07-302024-06-262025-10-13Bibliographically approved