DOI
https://doi.org/10.25772/M51W-7Z22
Author ORCID Identifier
0000-0003-2477-9971
Defense Date
2024
Document Type
Dissertation
Degree Name
Doctor of Philosophy
Department
Computer Science
First Advisor
Dr. Kostadin Damevski
Second Advisor
Preetha Chatterjee
Third Advisor
Dr. Bridget Thomson-McInnes
Fourth Advisor
Dr. Rodrigo Spinola
Fifth Advisor
Dr. Shadi Rezapour
Abstract
Emotions play an integral yet understudied role in open-source software development, profoundly shaping critical collaborative processes such as knowledge sharing, decision-making, and team dynamics. However, accurately detecting and analyzing emotions in developer communications poses significant challenges due to the lack of visual and auditory cues in text-based interactions. This dissertation investigates techniques to enhance the understanding and modeling of emotions within the textual artifacts of open-source projects. We conduct an extensive evaluation of existing emotion classification tools using a novel dataset of annotated GitHub comments. An error analysis reveals deficiencies in handling implicit emotional expressions and figurative language. We demonstrate that targeted data augmentation techniques can significantly enhance these tools, improving their performance on emotion classification tasks.
Next, we investigate the feasibility of utilizing Large Language Models (LLMs) such as BERT, RoBERTa, ALBERT, and CodeBERT for emotion classification. We find that they surpass domain-specific tools. Targeted improvements can be achieved through fine-tuning, improving the models' understanding of polarity and overall performance. However, our error analysis reveals that LLMs also struggle with figurative language. To address this, we conduct a study on software engineering-specific idioms and metaphors. By applying contrastive learning and task-specific fine-tuning, we enhance the models' ability to understand the context of figurative expressions. This approach shows consistent improvement in emotion classification as well as related tasks such as incivility classification and bug report prioritization.
Additionally, we experiment with the feasibility of utilizing zero-shot reasoning to automatically extract emotion causes from developer communications. By formulating suitable prompting strategies, generative Large Language Models like ChatGPT, GPT-4, and Flan-Alpaca exhibit reasonably accurate zero-shot performance on this challenging task. A case study uncovering common triggers of Frustration in TensorFlow issues showcases potential applications for gaining actionable insights. Furthermore, our research proposes an extension of the standard emotion taxonomy to better capture the diverse range of emotions expressed in software engineering texts, facilitating more accurate and nuanced emotion analysis.
Rights
© The Author
Is Part Of
VCU University Archives
Is Part Of
VCU Theses and Dissertations
Date of Submission
7-15-2024