Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Test-Driven Development Using LLM: A look into LLMs writing tests in a test-driven development workflow
Jönköping University, School of Engineering, JTH, Department of Computer Science and Informatics.
Jönköping University, School of Engineering, JTH, Department of Computer Science and Informatics.
2024 (English)Independent thesis Basic level (degree of Bachelor), 180 HE creditsStudent thesis
Abstract [en]

This research aims to explore different Large Language Models (LLM) and drawconclusions as to how well they can generate edge cases and respective unit tests in aTest-Driven Development (TDD) workflow. The experiment is concluded with LLMsthat are of the type Generative Pre-trained Transformers (GPT). The different LLMsthat are used for this research are ChatGPT 4 (GPT-4), ChatGPT 3.5 (GPT3.5) andGithub Copilot (Codex/GPT-4). The experiment consists of two phases, one where theprompt goes through prompt engineering with different methods such as few-shot andChain of Thought (COT). The second phase is where the prompt is sent to thedifferent LLMs to generate edge cases and unit tests. The prompt and questions thatare sent and asked to the LLM are taken from Advent of Code (AoC) which is aprogramming event every year. The LLMs performed differently depending on thetype of question from AoC. ChatGPT 3.5 performed better during text heavyquestions, ChatGPT 4 had an average performance throughout all the questions andGithub Copilot performed better towards the end where the questions became morecomplex and more programmatically focused.

Place, publisher, year, edition, pages
2024. , p. 51
Keywords [en]
Artificial Intelligence, AI, Test-Driven Development, TDD, prompt engineering
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:hj:diva-65413OAI: oai:DiVA.org:hj-65413DiVA, id: diva2:1878232
External cooperation
Combitech AB
Subject / course
JTH, Computer Engineering
Supervisors
Examiners
Available from: 2024-07-30 Created: 2024-06-26 Last updated: 2025-10-13Bibliographically approved

Open Access in DiVA

Test-Driven Development Using LLM(1828 kB)429 downloads
File information
File name FULLTEXT01.pdfFile size 1828 kBChecksum SHA-512
3d8f5dd923662fe4e84ee687e03e53bfab97198be3ca4ac48e2c215989efd4a92647324e9caf3d5e09862e37e91361cc5fc2dafe025a60bfc44eb80071b279ce
Type fulltextMimetype application/pdf

By organisation
JTH, Department of Computer Science and Informatics
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 429 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 1936 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf