Karas, Z.; Bansal, A.; Zhang, Y.; Li, T.; Mcmillan, C.; Huang, Y. “A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code Summarization.” ACM Transactions on Software Engineering and Methodology, Volume 33, Issue 7, 2024, Article 193, DOI: 10.1145/3664808.
Code summarization involves creating short, natural language descriptions of source code to help people understand it better. While previous research has looked at how programmers focus on different parts of the code when writing their own summaries, there hasn’t been much study on how they read and understand code with existing summaries. We don’t yet know how these two activities—reading and writing code summaries—compare, or how programmers pay attention to the meaning of the code during these tasks.
To explore this, we conducted an eye-tracking study with 27 participants to see where they focus when reading versus writing code summaries. We analyzed their gaze patterns, finding some differences in attention between the two tasks, as well as similarities in how they read code. We also noticed that factors like experience can influence these patterns. Additionally, we compared their gaze data to a structured representation of the code (Abstract Syntax Tree) and found that their visual focus doesn’t always match up with the actual code structure. These insights can help improve code comprehension in programming education and guide the development of automated tools for summarizing code.
.1. Example stimuli used in the task. In both conditions, the code was displayed on the left, and the summaries, pre-written or participant generated, were located in the top right. In the Reading condition, Likert scale questions for assessing summary quality were presented on the right below the pre-written summary.