CHAPTER II

METHODOLOGY

Kuder-Richardson Formula 20 Reliability Test.

Research Design

This study will employ a quantitative research design to assess the

effectiveness of AI-powered grammarly, in improving grammar proficiency of

college students. A descriptive survey design will be used to collect data from

participants. This design is chosen because it allows for the collection of

numerical data related to Grammarly’s impact on users’ grammar proficiency,

which can be analyzed to test the research hypotheses. This quantitative research

proceeded with the use of google form surveys to gather data.

Research Respondents

The respondents of this study will be composed of Electronics and

Communications Engineering (ECE) students enrolled at the University of

Mindanao. The selection of these respondents aligns with the study’s focus on

evaluating the impact of AI-powered Grammarly on grammar proficiency

among ECE students. Using the Raosoft Sample Size Calculator, the appropriate

number of respondents has been determined. With a confidence level of 95%, a

margin of error of 5%, and considering the total ECE student population of the

university, a sample size of 33 students is required for the study. This sample
size ensures adequate representation while maintaining statistical validity and

reliability.

The study was conducted at the University of Mindanao, located in Matina,

Davao City, Philippines. The University of Mindanao (UM) is a private, non-

sectarian university in Davao City that offers a wide range of undergraduate and

graduate programs, including the College of Engineering, which encompasses

Electronics Engineering.

Research Locale

The University of Mindanao was selected as the locale for this study because

of its reputation as a leading educational institution in the Davao Region,

offering accessible education to students from diverse backgrounds. It provides

an opportunity to assess the issue of uneven grammar proficiency among

Electronics Engineering students at the college level. The University of

Mindanao is also noted for its substantial population of students, many of whom

utilize AI-powered tools like Grammarly, making it an ideal setting to

investigate this phenomenon.

 Figure 1. Map of the Locale of the Study

Research Instrument

The research instrument that will be employed in this study is a questionnaire

developed by the researcher. This questionnaire is aimed at answering the

statement of the problem and will be the main source of the required data. It

shall have three sections: demographic information of the students, questions on

students using AI powered Grammarly, and an adapted multiple choice grammar

test assessing the levels of grammar competence of the students.

The profile section of the students’ survey will include the name (optional) and sex

of the participant. The checklist will seek to assess the students’ engagement with the use

of Grammarly. The modified multiple-choice test consists of: Subject-Verb Agreement,

Antecedents, and Prepositions. The examination the researchers intend to use will be a

modification of a published practice test by Alfarez, M. Z. (2020) entitled “NSAT and

College Entrance Test Reviewer, and Proficiency Test I for Incoming College Students of

TOP Universities like UP, Ateneo, De La Salle, and UST, and MSA Reviewer for ACET

(All College Entrance Tests) and UPCAT, and MSA Civil Service Reviewer for

Professional and Sub-Professional Levels.”

The research instrument will be evaluated for its reliability and validity based on

the Kuder-Richardson Formula 20. It is planned to perform a pilot study with college of

electronics engineering students of University of Mindanao, Davao City. In accordance

with the Kuder-Richardson Formula 20 reliability estimates it is expected that the

components of the test will include: Subject-Verb Agreement (moderate correlation),

Antecedents (moderate correlation), and Prepositions (high correlation). The item’s score

analysis shall make use of the index of discrimination and index of difficulty, in order

that only items of good quality are included, this is to demonstrate that the instrument is

ready for the last stage of the data collection.

As indicated by reliable sources, the goal of the researchers is to understand

thoroughly and appropriately the grammar knowledge of the students both before and

after the use of Grammarly. Items contained in the checklist which will be developed by

the investigators will evaluate the effect of Grammarly on the student’s grammar, its

usages, its advantages, and its effectiveness. The questionnaire will be developed in a

form of checklist type.

Before the commencement of the study, validation of each instrument and

reliability testing will be undertaken if needed ensuring that accurate responses to the

study question are made in respect of all instruments used. The researchers will submit a

first draft of their instruments to their supervisors for review where both formatting and

content relating to the variables of the study will be addressed. Revisions will also be

made based on feedback from advisors and members of the examining committee to

ensure that the structure, content and presentation of the instruments are appropriate.

Experts in the field will be asked to review the two sets of questionnaires in relation

to the objectives of the study. They will indicate their agreement on the statement using

check marks (✔) and single exes (X) or double exes (XX) It will be noted that some of

the concerns raised by the experts will be integrated into the final version of the

instrument. Items with inadequate percentage agreements will be either modified or

omitted altogether. Once all items are updated, all the items in the questionnaire’s

validator instruments developed by Good and Scates (1972) and further modified by

Abdullah and Cerado (2015) will be submitted to content validators for their various

reviews. What follows will be the determination of the validation criteria, rating scale,

and interpretation.
 Numeric Mean Score Quantitativ Descriptive Interpretation
al Rating Range e Rating
5 4.21 – 5.00 Strongly Strongly agree that each item of
Agree the indicator measures what it
intends to measure
4 3.41 – 4.20 Agree Agree that each item of the
indicator measures what it
intends to measure
3 2.61 – 3.40 Undecided Cannot decide that each item of
the indicator measures what it
intends to measure
2 1.81 – 2.60 Disagree Disagree that each item of the
indicator measures what it
intends to measure
1 1.00 – 1.80 Strongly Strongly Disagree that each item
Disagree of the indicator measures what it
intends to measure

After tallying and averaging the responses from three (3) content validators, the survey

questionnaire will be given an overall mean score. Please refer to the section below for a

detailed explanation of how to interpret the validation results.

Numerical Mean Description
Rating
5 4.21 - 5.50 Very Good
4 3.41 – 4.20 Good
3 2.61 – 3.30 Moderately Good
2 1.81 – 2.50 Poor
1 1.50 – 1.70 Very Poor
The instruments’ reliability for internal consistency will be obtained using the Kuder-

Richardson Formula 20 Reliability Test.

 Numerical Mean Verbal Description
Rating
5 0.91 – 0.99 Very High
Correlation
4 0.71 – 0.90 High Correlation
3 0.70 – 0.41 Moderate Correlation
2 0.21 – 0.40 Low Correlation
1 0.01 – 0.20 Negligible
Correlation
0 0.00 No Correlation

Research Procedure

The research was accomplished through the following processes:

I. Gathering of participants

The researchers gathered ECE college students as participants for the research. The

gathered participants were given a google form survey through online application.

II. Google form survey

A Google Forms survey will be designed to collect responses from participants. The

survey will consist of multiple-choice questions, Likert scale items, and demographic

questions. The questions will assess:

● The frequency and purpose of using Grammarly.

● The perceived improvements in grammar proficiency

● Time saved while using Grammarly.

● Overall user satisfaction with Grammarly.

● Types of errors most frequently corrected by Grammarly.

 Pre-Test: Before distributing the survey, it will undergo a pilot test to the participants.

This will help identify any unclear or ambiguous questions and ensure the survey is

functioning correctly.

III. handing out the survey

The survey will be shared with individuals who are either current users or have

experience using Grammarly. Target groups include Ece college students.

Channels for Distribution: The survey link will be distributed via:

● Email to university mailing lists, professional groups, or known Grammarly users.

● Social Media platforms, such as Facebook, LinkedIn, Twitter, or Instagram, where

participants can voluntarily opt-in.

IV. Informed Consent:

At the beginning of the survey, participants will be presented with an informed

consent page. They will be informed about the study’s purpose, the anonymity of

responses, voluntary participation, and the option to withdraw at any time.

Participants will need to agree to the informed consent before proceeding with the survey

V. Collection of Data

● Google Forms will automatically collect all responses, which will be stored in a

secure spreadsheet (Google Sheets).

● Responses will be stored anonymously, and no personally identifiable information

(e.g., name, email) will be requested or recorded unless given consent. The only

identifiable data will be responses related to writing habits, frequency of

Grammarly use, and satisfaction levels.

● Data will be collected and stored in real time, with participants able to submit their

responses 24/7 during the data collection period.

Key data points to be collected:

● Demographic Information: Age, occupation, education level (to analyze trends

across different groups).

● Grammarly Usage: Frequency of use (Daily, Weekly, Monthly, Never), type of

writing tasks (academic, business, etc.).

● Time Savings: Estimated time saved using Grammarly.

● Satisfaction Ratings: On a Likert scale (1-5) for error detection, ease of use, and

overall effectiveness.

● Error Correction: The types of errors most frequently corrected by Grammarly

(grammar, spelling, punctuation, style, etc.).

VI. Data analysis

● Descriptive Statistics: The first step in analyzing the data will be to calculate

descriptive statistics, which include:

○ Frequencies: The number of responses for each category (e.g., frequency of

Grammarly use, types of writing tasks).

○ Central Tendency Measures: Mean and standard deviation of satisfaction

ratings, time saved, and writing accuracy improvements.

Inferential Statistics:

● Paired T-test: To compare writing accuracy before and after using Grammarly, a

paired t-test will be used to identify any statistically significant difference in the
 quality of writing.

Reliability Testing: Cronbach's alpha will be calculated to assess the internal

consistency of the Likert scale items related to user satisfaction.

VII. Data Interpretation:

● After completing the statistical analyses, the results will be interpreted to

understand the relationship between Grammarly use and various factors (writing

quality, time savings, user satisfaction). Insights will be drawn on:

○ Grammarly improves writing accuracy.

○ The perceived time savings among users.

○ The level of satisfaction and the likelihood of recommending Grammarly to

others.

These insights will help in evaluating the hypotheses set out in the research and

contribute to the understanding of Grammarly’s impact on writing productivity

and accuracy.

VIII. Conclusion:

This research procedure outlines a systematic approach to investigating the impact

of AI-powered grammarly grammar proficiency among ECE college students.

Through a well-designed Google Forms survey, this procedure will allow for the

collection of reliable, valid data that can be analyzed to answer key questions

about Grammarly’s effectiveness in enhancing writing skills among college

students.

Statistical Treatment
 To determine the demographic profile of the respondents, descriptive statistics

such as frequency, and percentage will be used.

To determine the extent of student’s engagement and grammar proficiency, mean,

median, maximum, minimum, and standard deviation will be used.

To determine the significant difference in the AI-powered Grammarly engagement

between male and female students, a Cross tabulation, and Fisher’s exact test will be

used.

To determine significant difference in grammar proficiency between male and

female students, a t-test will be used.

To determine the significant effect of the engagement in the AI-powered

Grammarly on the grammar proficiency of the students, Chi-square will be used