Novice Debugging 2
Novice Debugging 2
By
Kavyansh Gupta
kavyanshlu@[Link]
ABSTRACT:
Background
Code familiarity refers to a programmer's ability to comprehend a code by recognizing its structure, logic,
and components. It can result from previous exposure to similar code patterns, functions, or algorithms. It
can also be considered a measure of the programmer's comfort with the code presented to them in terms
of code readability.
Objective
Inspired by the challenges faced by programming students, this study aims to investigate the influence of
a Novice programmer's familiarity with the code on the tactics they employed to debug it and their
effectiveness. Additionally, This study aspires to draw a comparison between the strategies applied by a
novice programmer and those preferred by expert ones. After a conclusion is made, the apex concern will
be resolved with the help of a programming project.
Method
Ten Introductory ( Novice) programming students with similar experience were given a Python code
fragment containing five international errors(bugs). They were asked to indicate on a scale of 1 to 5 how
familiar they were with the code along with the time taken by them to debug the code, the challenges they
encountered, and the tactics applied by them in carrying out the process. Based on their responses, they
were divided into groups of varying code familiarity concerning which the rest of the data was analyzed:
Results
Based on this analysis, Furthermore, it was found that the tactics used by the participants differed
depending on their level of familiarity with the code influencing the effectiveness. It was also found that
despite the different levels of participants' familiarity with the code, the challenges they faced were
similar. It was found that the number of bugs fixed was correlated with code familiarity and potentially
based on the tactics employed to debug the code. However, there was no consistent association between
the amount of time taken to debug the code, the tactics employed, and the familiarity levels.
Conclusion
Hence, these findings imply that improving a Novice programmer's familiarity with the code through
repeated exposure to similar patterns and structures can lead to more accurate and effective debugging
tactics. However, this does not lead to faster debugging or the reduction in the obstacles that may be
encountered. Furthermore, it was found that novices relied on surface-level features of the code and had
difficulties in understanding programming structures and syntax. Experts in this study, however,
employed more sophisticated methods of binary search, mental models, and hypothesis-driven debugging
Keywords
1. Code familiarity
1
2. Debugging tactics
3. Code comprehension
4. Effectiveness
5. Strategy analysis
6. Adept programmers
7. Novice Programmers
RESEARCH QUESTIONS :
[Link] are the challenges encountered by Novice programmers while debugging code?
[Link] tactics do novice programmers employ while debugging and how are they different than the ones
used by expert programmers?
[Link] does code familiarity influence the effectiveness of novice programmers in resolving bugs?
[Link] does the familiarity of code affect the time taken by novice programmers to debug it?
INTRODUCTION:
Literature Review
2
Helminen suggests that novice programmers face challenges in understanding and using different
programming environments, dealing with code fragment problems, and grasping specific programming
concepts such as collections, functions, error messages, iteration, outputting results, indentation, and
variables (Helminen et al.). Furthermore, the researcher noted that novice programmers tend to request
feedback less frequently when given execution-based feedback compared to line-based feedback,
suggesting that students may struggle with understanding the execution of their code and how to debug it
effectively (Helminen et al.). In continuation to previous studies, Juan D. Pinto suggested that novice
programmers struggle to isolate the problematic code or component causing bugs, fail to form robust
mental models of the problem and solution, and have difficulty explaining the purpose of code and
understanding its behavior in context (Pinto et al.). Additionally, the researcher found that novice
programmers may overlook security vulnerabilities in their code, as their priority is producing error-free
code rather than ensuring security (Pinto et al.).
In prolonging his analysis, Lahtinen reported that novice programmers employ varying tactics to debug
their code, for instance, adding a statement to print the contents of a variable and code tracing, which
allows the programmer to comprehend the step-by-step execution of the code (Lahtinen, Ala-Mutka, and
Järvinen 2005, 15-20). The researcher also observed that novice programmers use pattern matching,
where they change the code that "didn't look right", and isolation of code to discover the problematic area
through commenting out or altering the code (Lahtinen, Ala-Mutka, and Järvinen 2005, 15-20). Inversely,
Felix Lee argued that novice programmers employ tactics dealing with a dry run. For instance, testing
their code, taking notes on paper to process information more effectively, and asking people in their social
networks for help (Lee and Jones, 4). The researcher also concluded that novice programmers consider
their past experiences using particular debugging tactics and their experience in their debugging sessions
when deciding which debugging tactic to use (Lee and Jones, 4).
Pasquale Ardimento and his team highlighted the effectiveness of various debugging strategies, including
the reuse of known bugs, hypothesis generation, and strategic navigation (Ardimento et al.). The
researchers inferred that these strategies aid novice programmers in identifying and correcting errors in
their code, and also help them to develop problem-solving skills and critical thinking. Additionally, the
researcher reported that novice programmers can benefit from error detection and correction, marking,
and annotation, and strategic question categorization, ultimately succeeding in the debugging process
(Lin, Chang, and Chi). The researcher Manuel A. observed that tactics such as reading code, tracing
execution, debugging, and deconstruction techniques can also aid novice programmers in identifying and
correcting errors in their code (Pérez-Quiñones, Jordan, and Tucker, 15). For example, a novice
programmer may use code tracing to understand the execution of their code, as shown in the following
Python code:
def add_one(number):
return number + 1
def multiply_by_three(number):
3
return number * 3
print(add_one(multiply_by_three(5)))
Another successful strategy is the systematic approach to isolating the cause of a bug. According to Ko
and Myers, expert programmers tend to systematically localize a bug's location by using binary search
methods on the contexts of their codebase (Ko and Myers 2009). They continue to iteratively test sections
of the code until they can isolate the fault more efficiently. Another strategy is the use of complex mental
models of the software. Vessey noted that expert programmers have been known to develop and use
complex mental models of the software systems coded, which enable them to predict where the bugs are
likely to reside and what the effects are of various bugs (Vessey 1985, 459-494). Experts also rely heavily
on experiences and knowledge relating to common bug patterns. Lawrance, Burnett, and Dorn found that
experts have been known to engage in the use of reasoning analogies—that is, relating the current bug's
circumstances to previous problems they have debugged—to help them locate bugs more quickly
(Lawrance, Burnett, and Dorn 2013).
Experts also make heavy use of hypothesis-driven debugging. According to Fritz, Murphy, and Hill, this
process involves forming hypotheses about the manifestation of bugs, as well as their locations, then
testing them in a systematic process until either confirmed or denied (Fritz, Murphy, and Hill 2014). This
scientific basis of working allows for structured and effective debugging. Another important factor is the
use of debugging tools. According to Storey, Fritz, and Aranda, expert programmers have been known to
use highly productive debugging tools, such as IDEs, which enable them to trace and diagnose bugs more
effectively (Storey, Fritz, and Aranda 2014). Third-party documentation and resources are another factor.
Sillito, Murphy, and De Volder found that experts often refer to extensive documentation, forums, and
community databases to gain an aided insight and possible solution to a particularly challenging bug
(Sillito, Murphy, and De Volder 2006). For instance, Zeller has also found that debugging can be highly
collaborative. There is a tendency for experts to talk and share ideas with their peers so that they can
gather new insights and perspectives to facilitate the process of hunting down and squashing bugs more
quickly and effectively (Zeller 2009). Furthermore, expert programmers also make exhaustive use of
automated testing. According to Barr, Harman, and McMinn, "Automated tests will not only quickly
inform us whether bugs exist, but also often suggest where a bug is located, by failing and then telling
which test is failing and what values cause the failure" (Barr, Harman, and McMinn 2014). Another
strategy that expert programmers use is program slicing. According to Weiser, "Program slicing is a
method to take only part of a program that may affect some computation at some point (the slicing
criterion) into consideration, it facilitates tracking down a bug" (Weiser 1984). Calm and patient mental
attitude. This, harking back to the findings of Vessey, "Calming down one's anxiety and maintaining a
methodical approach is key to freeing up mental space for getting to the root of the problem" (Vessey
1985, 459-494).
4
When considering the debugging effectiveness of novices and experts in programming, two worlds
unfold. Corritore and Wiedenbeck state that the acquaintance of the code by a novice programmer will
always lean on code features that are closer to the surface (Corritore and Wiedenbeck 1991, 256-258).
This is because familiarity with the code by an expert programmer can structure the program by finding
semantic structures in the code so that they can reach a deep comprehension of what the code is doing and
fix bugs more easily and effectively as outlined by Pennington (Pennington 1987, 132-135). Studies
consistently indicate that a familiar code is important for effective debugging. For instance, Javidmanesh
et al. found that code familiarity is positively correlated with bug localization and fixing success, where
high rates of bug fixing are achieved when the code is familiar (Javidmanesh et al. 2013, 4). It further
showed that familiarity with the code guides comprehension, and effective bug identification, and
ultimately leads to bug-fixing achievement (Javidmanesh et al. 2012). Ehrlich et al. greatly emphasized
the connection of code familiarity to understanding, leading to the ability to effectively identify a bug
(Ehrlich et al. 2008, 412). The study by Khan et al. on the role of familiarity and code complexity in the
bug-fixing performance of programmers found that experts fixed bugs effectively, both in code that was
highly complex and in code that was normally complex, due to their extensively high familiarity with the
structures of the code (Khan et al. 2013). One of the reasons for the high
One of the reasons for the high debugging effectiveness of expert programmers is their efficiency in
localizing bugs.(Javidmanesh et a) found familiar code related to both faster and more accurate bug
localization, which consequently resulted in a higher number of resolved bugs. In general, the literature
proposes that expert programmers are more effective debuggers than novices based on their long
experience and familiarity with code structures, which translates to a better understanding of the code,
allowing the programmer to more rapidly identify and resolve bugs.
METHODOLOGY:
The methodology employed in this research paper involved the recruitment of approximately 10 students
with homogeneous levels of programming experience to participate in this study through a
pre-experimental questionnaire that collected data related to the participants' experience with Python
programming, willingness to participate in the study, basic concepts of python and hands-on ability in
running computer applications. Therefore It was assured that all the participants had all the necessary
prerequisite knowledge of Python programming and computer applications to contribute to carrying out
the procedure smoothly. A Python code containing 5 intentional bugs along with an algorithm of the code
was created based on the concepts of Python and the logical ability of the participants to comprehend the
purpose of the code with the help of the provided algorithm and its influence on the given code and bugs
in it. Before starting the
procedure, a quick and crisp overview of the study's methodology was given to participants. However, the
main idea of this research was kept un-told since the participants may adjust their natural course of
debugging the code to improve their performance which will hinder the final result. Throughout the
experimental procedure, meticulous records were kept of the participants' debugging processes, including
the time taken, strategies employed, and familiarity Following the completion of the debugging tasks,
participants were invited to provide feedback through a post-experiment questionnaire, assessing their
perceived difficulty, confidence levels, and overall satisfaction with the debugging process. The data was
analyzed in-depth by both descriptive and inferential statistical analyses to compare if the time spent,
5
strategies employed, and challenges faced, across different levels of code familiarity, showed any
difference in debugging efficiency as identified by the number of bugs accurately resolved. Considering
these factors, the participants were divided into two groups, Group A which comprised less adept novice
programmers who had a familiarity of 1 to 3 on a 5-point scale, and Group B which comprised slightly
more adept programmers who had a familiarity of 3.5 to 5 on a 5-point scale. for more in-depth analysis,
participants having identical familiarity with the code were kept in the same sub-group. Now, the average
of the time taken and the number of bugs resolved by each group were calculated using the following
formulas :
Average time taken =sum of the time taken by the participants falling in that group number of
participants falling in that group
Average no. of bugs resolved =sum of bugs resolved by all participants falling in that group number of
participants falling in the group
This data was later used to compare strategies employed by novice programmers with varying levels of
familiarity with the code.
Count = 2
While count < age:
Count == 1
print("Are we there yet?")
6
for i in range(1 , age + 1):
If age % i == 0
print(i, "is a factor of your age")
Input("End of Program")
Provided Algorithm :
Check if age is greater than or equal to 21 and print eligibility for alcohol consumption
Check if the age is greater than or equal to 18 and print eligibility to vote.
Display the factor of the user's age using a loop
Print an extra statement at the end of the program.
Intentional bugs :
[Link] 8: A missing colon after "else" (Structural syntax error)
[Link] 10: A missing quotation mark after the print statement, before the bracket (structural syntax error)
[Link] 14 : 0 instead of 2 ( Algorithm interpretation and critical thinking)
[Link] 16 : '>=" to come instead of "==" ( conceptual and critical thinking")
[Link] line: Print to be used instead of Input.
Rectified code :
While True:
Age = int(input('Enter your age:'))
break
Except ValueError:
print('Invalid Input. Please enter a valid number)
Count = 0
While count < age:
Count >= 1
print("Are we there yet?")
7
else:
print("You are legally not allowed to consume alcohol")
print("End of Program")
RESULTS:
8
Tactics and strategies used to debug the code
Participants from Group A took an all-round approach to debugging the code. They first started with a
grand strategy of reading the code. They skimmed through the whole codebase, taking heed of details, and
tried to trace any blinding errors or blemishes. Moreover, they got back to the old-school technique of pen
and paper to take a red run at how the code would execute. This allowed them to simulate the program's
flow step by step, gaining an in-depth understanding of its behavior and where potential pitfalls may
occur. Members of Group A were further determined to try and relate the code to some basic Python
concepts, which ensured that the code followed the syntax and semantics of the Python language. This
certainly helped to shorten the time for detecting and fixing errors since they could find areas in which
they differed from the expected norms by relating the code to the fundamental principles. In addition to
this, they compared the problematic code with known working examples. This comparison would let them
identify discrepancies and inconsistencies that guided their debugging in a much more effective way. In
addition to this, Group A members were always keen on seeking opinions and feedback from their peers,
which allowed ideas from others to easily be considered in the debugging process. Here, they capitalized
on the collective competence and experience of their colleagues to find some hidden issues and explore
alternative solutions. Finally, they remained watchful for common syntax errors, such as indentation
problems and faults of a typographical nature, which are an eyesore to most. By keenly looking at these
common pitfalls, they managed to correct them quickly, thus retaining the code's integrity.
In contrast, Group B participants were methodical and broke the code into chunks they felt were
digestible and would not overwhelm them while analyzing it. This strategic move helped them pay close
attention to the sections of the code, which thus enabled closer inspection to be performed concerning the
logic and function of the code. They checked every piece of the code against the algorithm provided,
taking meticulous care to check if there was any deviation or disparity within the algorithms that could
potentially form a fault. Additionally, members of Group B used the strategic addition of print statements
to check the contents of variables and the flow of execution. By strategically placing such diagnostic tools
at vantage points within the code, they could get valuable knowledge of its operation as well as possibly
obtain the point of failure. They also applied the concept of isolation, where they commented out lines of
the code to observe the behavior of the program. Such an isolated approach helped in reducing the
problematic area, therefore narrowing down the root cause of the problem more effectively. Group B also
took the help of experiences from previous encounters with such codes and debugging sessions and used
the knowledge to strategize their approach to debugging. With previous encounters to draw from, they can
base decisions on a plethora of knowledge, using tactics most effective in the face of the unique problems
that arise. Conclusion Participants from both Groups A and B demonstrated impressive diligence and
resourcefulness with their strategies for debugging and managed to combine a blend of analytical skills,
collective effort, and strategic thinking toward uncovering and resolving issues
DISCUSSION:
Interpretation and analysis of results
9
In this section, we first, attempt to compare the debugging tactics of the novices as in this study and, to
some extent, those of the experts given their familiarity with the code, concerning established results of
past studies. We also hoped to correlate the same tactics against, first, the challenges that they
encountered in debugging the code, and second, the amount of time spent on debugging the code and how
effective the tactic was, as measured by the number of bugs resolved. The debugging tactics novices
employed in our study—such as code tracing and printing variable contents—were also followed by
earlier research, which concluded that novices relied on surface-level features of the code and had
difficulties in understanding programming structures and syntax. Experts in this study, however,
employed more sophisticated methods of binary search, mental models, and hypothesis-driven debugging,
which is also per established results: experts build complex mental models of the software systems and
use these mental models to predict where faults are likely to be located. The correlation of code
familiarity with debugging effectiveness is also accorded in prior research, which suggests that bug
localization and fixing success are positively correlated with code familiarity. The challenges that novices
face during bug fixing, such as "difficulties in understanding programming structures and syntax", are
also consistent with the dilemmas committed by novices on features of programming, which divide
functionality into procedures. Moreover, the observation that higher-familiarity experts still invested a
similar amount of time in debugging their programs as lower-familiarity novices suggests that while
higher code familiarity may indeed lead to more effective debugging, it may not necessarily result in
faster debugging times, corroborating the existing view suggesting that experts may be more careful and
thorough in their debugging approach . In this respect, the results of this study support the idea that
debugging represents a complex cognitive activity that interacts with several factors among them: code
familiarity, debugging experience, and individual differences in strategies for problem-solving . The
improved performance of higher-scorers within the novice population {i.e., score 3}suggests that even
small to moderate amounts of code familiarity could be of considerable advantage for debugging results,
reconfirming previous findings that code familiarity can be augmented through practice and experience .
Again, this finding demonstrates the challenge taken into account while designing debugging tools and
environments that support effective debugging practices in this complex interplay between code
familiarity, debugging experience, and individual differences in problem-solving strategies.
The results of this study also indicate that the relationship between code familiarity and effectiveness at
debugging is more complex and multi-faceted. Although higher code familiarity is associated with better
debugging performance, there may be other influencing factors on the debugging process. For example,
code complexity, the nature of the bugs, and the experience and skills of an individual in debugging of the
code may all have noticeable effects on the strategies for good results. Moreover, the findings of the study
suggest that debugging could be done using different tactics and strategies, where experts use the mental
models of prediction for the most preferable places for the bugs to be located. This is confirmed by
research that states that experts build complex mental models of software systems and use them to direct
their debugging efforts
It can be inferred from this study that, in general, the role code familiarity plays in debugging
effectiveness has been confirmed, and that both novices and experts use different tactics and strategies
during the debugging task, which are conditioned by different degrees of code familiarity. More
importantly, the study found that such consideration of complex interactions between code familiarity and
debugging experience with individual differences in problem-solving orientations are necessary
prerequisites when designing tools and debugging environments that can foster effective debugging
among both novices and expert programmers. The contribution of the results of this study is therefore the
10
enhancement of further research in the area of involved cognitive processes, which may indeed lead to the
development of both more advanced debugging tools and environments that can effectively support
novice and expert programmers.
Limitations
It was difficult to find previous studies that directly dealt with the concept of code familiarity. Code
familiarity is a highly underused term that can be comprehended and used in many different ways. This is
probably the main reason why there is not much literature that deals with different levels of code
familiarity and how they affect programmers' efficiency while debugging. Extending this line of research
could provide valuable insight into how familiarity with code positively and negatively influences a
developer's ability to debug code. Also, to extend upon this line of research, currently, there are no studies
that investigate how developers maintain and build familiarity over time and how this concept relates to
other variables such as team structure and the size and complexity of the codebase. Thus, the current
literature surrounding code familiarity suggests that there is a need for more targeted empirical studies
aimed at understanding the positive and negative effects of code familiarity on a developer's debugging
abilities.
CONCLUSION:
After critical analysis and interpretation of the results, It can be concluded from this study that, with
repeated exposure to similar code patterns, functions, or algorithms, one might be able to increase their
accuracy in the process of debugging a code but it does not guarantee a reduction in the amount of time
taken to do so. Also, we could assess the vast difference between the tactics applied by novice
programmers and those applied by expert programmers in this process. it was found that novices relied on
surface-level features of the code and had difficulties in understanding programming structures and
syntax. Experts in this study, however, employed more sophisticated methods of binary search, mental
models, and hypothesis-driven debugging, which is also per established results: experts build complex
mental models of the software systems and use these mental models to predict where faults are likely to
be located. Also, Novice programmers who were more adept dealt with the debugging process with a
technical approach but the ones who were less adept relied on basic methodologies dealing with core
concepts and dry-runs. Also, more adept programmers relied on their past experiences while choosing a
debugging strategy. More importantly, the study found that such consideration of complex interactions
between code familiarity and debugging experience with individual differences in problem-solving
orientations are necessary prerequisites when designing tools and debugging environments that can foster
effective debugging among both novices and expert programmers. The primary challenges faced by
novice programmers were found to be that they misinterpret some minor aspects of code, struggle with
complex code structures, and find algorithms perplexing, leading to anxiety that hinders their
performance. However, with higher familiarity, novice programmers were able to analyze potential errors
more efficiently and therefore resolve more bugs, although they still encounter anxiety due to vast code
length which can be overcome eventually by repeated exposure to similar codes.
11
ACKNOWLEDGEMENT
I wish to extend my gratitude to Ms. Prem Bala (PGT Computer Science), Ms. Nisha Aneja (Senior
School Incharge), and G.D. Goenka Public School, Sector 22, Rohini, New Delhi for providing me with
support and resources that made the conduction of this research experiment possible in their computer lab
with their guidance throughout the project. I am obliged to my peers who participated in the experiment,
which remains very central to the success of the project. Moreover, I am very grateful to the School
Principal, Dr. Vandana Gupta, for the approval and support that enabled me to carry out this study.
REFERENCES:
Ardimento, Pasquale, Mario Luca Bernardi, Marta Cimitile, and Giuseppe De Ruvo. "Reusing Bugged
Source Code to Support Novice Programmers in Debugging Tasks." ERIC.
Barr, E., M. Harman, and P. McMinn. "The Oracle Problem in Software Testing: A Survey." IEEE
Transactions on Software Engineering, 2014.
Corritore, C. L., and S. Wiedenbeck. "What Do Novices Learn During Program Comprehension?" 1991,
256-258.
Ehrlich, Susan W., et al. "Expertise Remakes Debugging Strategies." 2008, 412.
Fritz, T., G. Murphy, and E. Hill. "Does a Programmer's Activity Indicate Knowledge of Code?" ACM
SIGSOFT Software Engineering Notes, 2014.
Helminen, Juha, Petri Ihantola, Ville Karavirta, and Lauri Malmi. "How Do Students Solve Parsons
Programming Problems? — An Analysis of Interaction Traces."
Javidmanesh, Mohammad, et al. "How Does Code Familiarity Impact Bug Localization in
Programming?" 2013, 4.
Javidmanesh, Mohammad, et al. "How Familiarity with Code Affects Bug Localization During
Debugging." 2012.
Khan, Sanaullah, et al. "The Effects of Familiarity and Code Complexity on Programmers' Bug Fixing
Performance." 2013.
Ko, A., and B. Myers. "Finding Causes of Program Output with the Java Whyline." ACM SIGCHI
Conference on Human Factors in Computing Systems, 2009.
12
Lahtinen, E., K. Ala-Mutka, and H. M. Järvinen. "A Study of the Difficulties of Novice Programmers."
2005, 15-20.
Lawrence, J., M. Burnett, and B. Dorn. "To See or Not to See: The Effectiveness of Code Highlighting for
Rapid Defect Detection." IEEE Transactions on Software Engineering, 2013.
Lee, Felix, and James A. Jones. "Exploring How Novice Programmers Pick Debugging Tactics When
Debugging: A Student's Perspective." eScholarship.
Lin, Tzu-Ying, Chia-Hui Chang, and Yu-Lin Chi. "The Investigation of Novice Programmers' Debugging
Behaviors to Inform Teaching Practices." ERIC.
Pérez-Quiñones, Manuel A., Kris Jordan, and Colin R. Tucker. "Learning by Taking Apart:
Deconstructing Code by Reading, Tracing, and Debugging." Academia.
Petersen, Andrew, Michelle Craig, and Daniel Zingaro. "Software Debugging Patterns for Novice
Programmers."
Pinto, Juan D., et al. "Investigating the Relationship Between Programming Experience and Debugging
Behaviors in an Introductory Computer Science Course."
Sillito, J., G. Murphy, and K. De Volder. "Questions Programmers Ask During Software Evolution
Tasks." ACM SIGSOFT Symposium on the Foundations of Software Engineering, 2006.
Sorva, Juha, and Arto Vihavainen. "A Think-Aloud Study of Novice Debugging."
Storey, M.-A., T. Fritz, and J. Aranda. "How Do Software Developers Use Code Annotations? A Case
Study." IEEE Transactions on Software Engineering, 2014.
13