Question Template Labeling

A. Project Introduction
What is question template labeling?

● Project goal: To build a high-quality & dynamic homework problem library that serves the study of college
students in STEM subjects, including: Algebra, Calculus, Statistics, Trigonometry.

● Question template labeling: is the process where experts contribute to the development of high-quality question
templates that meet the project pre-defined expectations.
○ Question template: is a dynamic question that covers a certain math knowledge with the ability to
generate different variables of the same question to each different student. This allows students to receive
a question from the same topic but with different inputs to prevent potential cheating.
○ A question template must be designed based on the knowledge taken from a given Learning Objective
(LO) from a predefined textbook, and/or based on a sample textbook exercise .
■ A textbook is normally presented with the following structure: Chapters > Units > Learning
Objectives.
B. How To Label A Template
B. Template Overview

1. What is a template?
Each template represents an exercise in the homework problem library and is designed to generate question variations
at the LO level, allowing for dynamic and randomized content.

● This is how a template

displays in our labeling portal
- under the expert’s view.
● Each component of a
template has a preview box
on the right (except the
common control & comment
section). This helps us
ensure the template displays
well on user’s portal, and
helps your labeling process
become easier.
B. Template Overview

2. Main components of a template

There are 5 main components in a template, which are: (1) Description, (2) Common Control, (3) Question, (4)
Explanation, (5) Comment (Note: Review slide 12 for further explanation of these 5 components)
B. Template Overview

3. Template type
- There are 2 types of question templates:
- Pre-generated templates: The template is pre-filled with content for (2) Common Control & (3) Question
parts. The labeler work is to review and edit the pre-filled content then add the (3) explanation & other parts
to make it meet the labeling requirements.
- Blank templates: The template has NO pre-filled (2) Common Control & (3) Question parts. The labeler
work is to develop the question & common control themselves, then add the explanation and complete
other parts as normal.
B. Labeling Requirements

Requirements of a qualified templates

● A qualified template must meet the requirements in these criteria:
○ Accuracy: The template must be dynamic & mathematically correct.
○ Flow: The template’s explanation is detailed & well-organized.
○ Format: The template is well presented with no typo/grammar/format mistakes/issues.
B. Labeling Requirements - Accuracy

Overall The question The explanation

- The template is relevant to the If the question is NOT pre-generated but - The generated step-by-step

chosen Learning Objective (LO) built based on a specific textbook explanation follows the
- The question and explanation are problem: textbook-solving method
mathematically correct. - The question is fully paraphrased

- The template is dynamic, ensuring with no direct copying or minimal

the question and explanation changes

remain valid when inputs/variables - The graph structure is significantly

are regenerated different from the textbook (e.g.,

different function types, altered
slopes, discontinuity points,
adjustments in the number and
positioning of graphs, etc.).
- The question flow is clearly
different from the textbook while still
maintaining logical progression
B. Labeling Requirements - Flow

The Explanation has a qualified Flow when it:

● Follow the steps as taught in the textbook.
● Start the steps with a verb, summarizing the purpose of that step.
● Be clear & easy for the student to understand the question and fill in the correct answer (E.g if the question requires
graphing, there must be clear graphing requirements, etc.)
B. Labeling Requirements - Format

A template with a five score will:

- Use proper English with no typos or grammatical errors.
- Follow strictly the standard format for writing explanations.
- All math notation must be written in AsciiMath
 B. How To Label A Template -
Detailed guidelines for each component
B. How To Label A Template - Overall steps

1. Review the Textbook Knowledge

● Review the relevant content from the assigned Learning Objective (LO) in the textbook.
● Confirm that the template is assigned to the correct LO.
2. Label Template Components
- If the template is blank, construct the Question & the Common control. Otherwise review the content from the pre-generated
Question & Common Control.
- Generate & review the Explanation:
- Click “Generate Explanation” to produce an initial explanation for the question
- Edit the explanation to ensure it follows the detailed guidelines
● Ensure the template is dynamic
○ Click the “Generate Randomized Preview” button at least 5 times to test the template. Look for any inconsistencies or
inaccuracies in the explanation. Examples of issues:
■ A value could be either positive or negative, changing the logic of the explanation.
■ An expression may appear in an odd form (e.g., 1x, x^0, x^1).
● Recheck/ Add information for the Description & Comments components
3. Review the Labeled Template
● Click “View as Student” to verify template behavior.
○ Try a correct solution and view the explanation — repeat at least 5 times to ensure the explanation works across
different values.
■ If inconsistencies are found, revise the explanation and common control as needed.
○ Try an incorrect solution to confirm it is properly rejected.
4. Submit Your Work
● Once everything is reviewed and works as expected, submit the labeled template.
B. How To Label A Template - (1) Description

1. What is the description for?

● The template description is a section that provides a brief summary of the template’s source and
content. While it is not part of the labeling task, it is a helpful reference for the internal team and
reviewers to quickly understand what the template is based on.
● This section typically includes:
○ The template category (pregenerated or blank templates, etc.)
○ A short summary of the template’s main topic or concept
○ The corresponding exercise number from the textbook (e.g., "Section 3.1, Exercise 12")
This information helps track coverage across learning objectives and avoid duplicates.

2. How to draft the description

● This section is normally pre-filled by the team before being assigned to experts. However, if the
description is missing a brief summary of the template’s main topic or concept, please add it yourself.

3. Example of a good description

● Please refer to the next slide.
B. How To Label A Template - (1) Description

The location of the corresponding A brief description of the exercise

exercise in the book type - making it easy for us to
identify suitable templates

Example of description in a template

B. How To Label A Template - (2) Control Code (or Common Control)

1. Control code definition:

● The control code (also referred to as the common control) is the section of a template that defines all
variables, constants, and calculations required for the question and its explanation.
● Its primary purpose is to ensure that the template is both dynamic (able to generate multiple valid
instances with different values) and accurate (mathematically consistent and logically correct).
● The control code plays a key role in determining the accuracy, and in some cases, the formatting
structure of the question and explanation.

2. How to draft control code for a template:

● We base the creation of question control codes on this guideline. A more detailed & relevant
instruction will be provided if you join the project.

3. Some examples of control code in a template:

● You can refer to the next slide to see more examples of control code in a template.
B. How To Label A Template - (2) Control Code (or Common Control)

Define the type of

the question

Set rules for the

answer/question

Example of control code in a template

B. How To Label A Template - (3) Question

1. What is a template question?

● The template question is a reworded version of an exercise taken from the textbook, designed to
align with a specific learning objective.
● While it is based on the original problem, the question must be fully paraphrased following the
project’s guidelines—this includes changing the wording, numbers, and context to avoid direct copying
● In addition, the template question must be dynamic, meaning it should work with variable inputs from
the control code to generate multiple variations while preserving the original mathematical concept.

2. How to draft questions for a template:

● The draft question must follow the requirements as stated in Slide 09 & 10.

3. Some examples of questions in a template:

● You can refer to the next slide to see more examples of questions in a template.
B. How To Label A Template - (3) Question

The question contains text & variables from the

control code
This preview box allows you
to see how the question
The values in red represent the variables in the appears under user's view
question, which makes the template dynamic.

Example of question in a template

B. How To Label A Template - (3) Question

This preview box allows you

to see how the question
The question contains text & variables from the appears under user's view
control code, however, this is NOT a good question
since it has no variables, which makes the template
hard-coded.

Example of question in a template

B. How To Label A Template - (4) Explanation (or Step-by-step Solution)

1. What is a template explanation?

● The template explanation provides a step-by-step solution to the template question. It should follow
the problem-solving approach used in the textbook, ensuring that the logic, method, and
terminology are consistent with what students are expected to learn.
● The explanation must also be dynamic, meaning it should accurately respond to the values generated
by the control code and remain logically valid across different variations.
● A strong template explanation is clear, well-structured, and helps guide students through each
step required to solve the problem.

2. How to draft an explanation for a template:

● The explanation must follow the requirements as stated in slide 09.

3. Some examples of explanation in a template:

● You can refer to the next slide to see more examples of explanations in a template.
B. How To Label A Template - (4) Explanation (or Step-by-step Solution)
The explanation contains text & variables from the control The values in red represent the variables in
code the question, which makes the template
dynamic.

This is a good example of

an explanation: divided
into steps with a clear
goal in each step.

Example of explanation in a template

B. How To Label A Template - (5) Comment

1. What is the comment section for?

● The template comment is a dedicated section used for communication between experts and the
internal team. It is not a part of the template that needs to be labeled or finalized, but it plays an
important supporting role during the labeling and review process.
● Experts should use the comment section to leave important notes about the template, such as:
○ Which explanation flow was followed (Openstax, current textbook, or own draft - e.g.: Flow -
[Own draft])
○ Any concerns or edge cases identified during the labeling process
○ Clarifications about why specific changes were made
● The internal team may also use this section to leave instructions, feedback, or notes related to the
template.
● Note: Comments should be concise, relevant, and helpful for reviewers and auditors.

2. How to leave proper notes in the comment section:

● Details & examples on how to leave notes in this section can be found in the next slides.
B. How To Label A Template - (5) Comment
Here are pieces of information that need to be included in the comment section
a. Question type (mandatory): Since we haven’t had the function for experts to change the type of questions for certain
templates, we will need the help of experts to note down the actual question type in the comment section, whether it is
MCQ (multiple choice), MSQ (multiple select), or FITB (fill in the blank).
The structure for your notes should be: Type - [Actual type of that question]
For example: Type - [FITB]
b. Explanation flow (mandatory): Follow the detailed instructions in the guidelines for Explanation in slide…
c. Other notes (optional):
○ Elaborate notes: These notes only help auditors or the internal team to better understand the template, or your
labeling approach. They should not affect the experts’ ability to label this template. For example, “I adjusted
part c to fit the LO better”, “FITB probs are not suitable for this LO”, etc.
The structure for your notes should be: [Elaborate] - Your notes
For example: [Elaborate] - FITB probs are not suitable for this LO
○ Software issues: There might be functions that haven’t been supported on our portal yet and prevent you from
labeling templates in a specific LO. In that case, please leave a note in the comment section so our internal
team can be aware of such issues. For example, “Cannot view the draw”, “There’s no tool for mixed numbers”,
etc.
The structure for your notes should be: [Software issue] - Your notes
For example: [Software issue] - There’s no tool for mixed numbers
B. How To Label A Template - (5) Comment

A template with proper notes should have the comment section like this:
C. Practice
B. How To Label A Template - (5) Comment

You can practice our labeling steps before taking the relevant test as follow (Follow the step in slide 13):
- Log into the labeling portal using this account:
- User name: practice@[Link] / Password: Mathgpt2025!
- Take a look at these qualified sample labels: Exercise 46, Example 02
- Practice the following instructions yourself using:
- This blank templates: Exercise 52,
- Textbook reference: please locate to exercise 52 of Unit 4.1 Basic of Differential Equations (Image)
- Make sure to clear your practice content after you're done with your practice session.