Intent
Classification
[Workflow Walkthrough]
1
Purpose
To teach models to recognize what the customer is trying to do or ask in an
e-commerce scenario, based purely on their message (intent-level
understanding).
Intent classification means identifying what a customer is trying to
accomplish based solely on their message. It focuses on the customer’s
underlying goal, such as asking about a price, checking an order, or
requesting a return, regardless of how the message is phrased
Page 2
• Intent classification identifies the customer’s goal from their message.
• You will create a realistic mini-conversation ending in a final customer
message.
• Your task is to classify that message and explain the reasoning.
• Focus on selecting the primary intent when messages are mixed or
ambiguous.
• Always follow metadata requirements, including product domain, tone, and
constraints.
Page # Page 3
Page # Page 4
Page # Page 5
Intent
Intent Category Definition Intent Category Definition
Product_Information_Request Product details or specs Account_Issue Login or account access problems
Price_Inquiry Current price or Recommendation_Request Product suggestions or guidance
discounts
Categories
Availability_Check Whether item is in stock Discount_Inquiry Sales, promos, or coupon questions
Comparison_Request Compare features or Cancellation_Request Cancel an unshipped order
products
Return_Policy_Question Return rules or process Exchange_Request Swap item, size, or color
Warranty_Inquiry Coverage or claims Gift_Option_Inquiry Gift wrap or card inquiries
process
Order_Status_Check Track or check order International_Shipping_Question Shipping abroad or region-specific
status
[Ground
Product_Complaint Report defective or Bulk_Order_Inquiry Large quantity or wholesale requests
poor-quality item
Feature_Inquiry Specific product feature Customization_Question Personalizing or customizing products
inquiry
Compatibility_Question Whether product works Subscription_Management Managing recurring or auto-ship orders
Truth]
with another
Size_Guidance_Request Fit or sizing help Product_Restock_Inquiry Ask when out-of-stock item returns
Installation_Help Setup or installation help Store_Location_Inquiry Physical store location or hours
Troubleshooting_Request Fixing or diagnosing Loyalty_Program_Question Rewards or membership benefits
product issues
Payment_Issue Billing or payment Technical_Specification_Question Deep technical or spec questions
problems
Shipping_Question Delivery, cost, or Authenticity_Verification Checking product authenticity
timeframe
Other None of the above
Page 6
What You’re Doing
You’ll read a short scenario and its metadata, then building a high-quality example of a conversation between user and agent
based on the scenario and metadata.
The full conversation should be from scratch, without use of LLM. This includes:
● Writing the Context : The conversation turns that happened before the customer’s last message. Context = the earlier
user/agent turns + background info that lead directly into the customer’s Input.
● Writing the user’s final message (Input): a natural, professional, technically specific user utterance that fits the
scenario.
● Writing the Reasoning Trace (RT): a clear, step-by-step explanation of how the agent should arrive at the correct
answer for the Ground_Truth intent.
Page # Page 7
Why the components matter
● Context ensures the model learns to interpret intent within realistic multi-turn
conversations.
● Input is the actual message being classified and must stand on its own.
● Reasoning Trace teaches the model how to logically determine the intent, resolving
ambiguity when needed.
● Ground Truth provides the single best intent classification based on the customer’s
goal.
Page # Page 8
What You’re
Doing 2.
Review the
3.
Write context
metadata (user-agent turns
before last prompt)
1.
Read the scenario
4.
Write input (last user
5.
Write a brief
prompt) reasoning trace
(3–7 steps) that
justifies the label.
Page 9
Scenario Step 3 | Write Context Notes
Write a 2-turn conversation between a user and Check the Context against the Quality Metric
customer service agent. The customer's intent User: hey I still didn't get loyalty points for my BIC ● E-commerce Interaction - yes, user reaches out to the
must be Loyalty_Program_Question. Client has gel pens order. also they still didn’t show up and it chatbot on ecommerce site where they placed the order
not received the product within the duration says customs is like holdin them?? 😭😭 ● Realistic Conversation - yes, this is a realistic scenario, you
promised. Include 2 emojis and 2 typos Use Agent: Thank you for reaching out. I can confirm the can imagine a real person having such problems with the
delay is due to an extended customs inspection, but order
filler words and genz slang. Include details ● Natural Conversational Flow - yes, the dialogue is logical,
about delays caused by customs. your order is still moving through the process and
coherent, and easy to follow.
should be released in the next day or two. Your
● Agent Inputs are free of Grammar Errors, Typos,
Metadata loyalty points will be credited once the order is Colloquial Language - yes, agent’s inputs are professional,
finalized. grammatically correct, and free from informal expressions.
Ground_Truth: Loyalty_Program_Question User: ok bet, just making sure. Customs realy be ● Agent is Truthful - yes, agent provides factual, accurate and
Category: Order_Management consistent information with no contradictions.
moving slow fr fr
Product_Category: Office & School Supplies ● Agent is Helpful - agent’s responses directly address the
Issue_Type: Delivery delays
Agent: I understand the frustration. Customs delays
customer’s request, providing complete and useful
Cultural_Context: Include details about delays caused have affected multiple shipments this week, but
information.
by customs yours should clear soon. Your loyalty benefits ● Real Product - yes, BIG gel pens is not real product,
Language_Style: Informal language remain fully intact changed to the real BIC gel pens
Linguistic_Complexity: Casual ● Match Metadata - yes, all metadata requirements are
Complexity_Level: Moderate fulfilled
Product: BIG gel pens ● Match Ground Truth
WRITING CONTEXT
Example
Page # Page 10 10
What is a “turn”?
Turn = User message + Agent message (together count as 1 turn).
Example:
Turn 1: User message → Agent response
Turn 2: User message → Agent response
Page # Page 11
Scenario Step 3 | Context Notes
Write a 2-turn conversation between a user and User: hey I still didn't get loyalty points for my BIC gel pens order. also
they still didn’t show up and it says customs is like holdin them??
customer service agent. The customer's intent 😭😭 Check the Input against the Quality Metric
must be Loyalty_Program_Question. Client has Agent: Thank you for reaching out. I can confirm the delay is due to an
not received the product within the duration extended customs inspection, but your order is still moving through
● Logical and Realistic Continuation of
the process and should be released in the next day or two. Your
promised. Include 2 emojis and 3 typos Use loyalty points will be credited once the order is finalized. Context: the input is a natural next line in
filler words and genz slang. Include details User: ok bet, just making sure. Customs realy be moving slow fr fr the conversation coming from User, right
Agent: I understand the frustration. Customs delays have affected
about delays caused by customs. after the context. It must be consistent with
multiple shipments this week, but yours should clear soon. Your
loyalty benefits remain fully intact a real-life e-commerce scenario.
Metadata ● Match Metadata: Input fully matches
metadata.
Ground_Truth: Loyalty_Program_Question ● Match Ground Truth
Category: Order_Management
Product_Category: Office & School Supplies
Step 4 | Input
Issue_Type: Delivery delays
Cultural_Context: Include details about delays caused gotchu, thanks! and just to dbl check, when
by customs you say loyalty points will be credited once
Language_Style: Informal language the order is finalized you mean when the
Linguistic_Complexity: Casual pens arrive? or when the customs clears?
Complexity_Level: Moderate
Product: BIG gel pens
WRITING INPUT
Example Page # Page 12 12
STEP 5
Reasoning Trace
This is a step-by-step explanation of how the agent should arrive at
customer’s intent.
This explanation should be procedural, without fluff.
● Contain 3-7 logical steps.
● Show clear progression from input analysis to conclusion.
● Highlight key information and its relevance.
● Explain any ambiguities and how they are resolved.
● Demonstrate the logical decision-making process.
Team Name Page # Page 13 13
How to handle ambiguous intents
When a message contains multiple topics, identify the customer’s
primary goal.
● Prioritize direct questions over side comments.
● If the user expresses frustration but asks a specific question, classify the
question.
● When two intents are possible, use the reasoning trace to justify the choice.
Team Name Page # Page 14 14
Step 3 | Context Step 5 | Reasoning Trace Notes
User: hey I still didn't get loyalty points for my BIC gel pens Check Reasoning Trace against the Quality Metric
order. also they still didn’t show up and it says customs is
1. The user states they didn’t get loyalty points
● 3-7 Logical Steps: 5 steps
like holdin them?? 😭😭 for their BIC gel pens order, indicating a
● Clear Progression from Input Analysis to Conclusion:
Agent: Thank you for reaching out. I can confirm the delay direct inquiry about the loyalty program. reasoning demonstrates a logical analytical sequence of
is due to an extended customs inspection, but your order is 2. The user also expresses disappointment input and context (with the main focus on input) leading to
still moving through the process and should be released in with delivery delays caused by customs, but the final decision - ground truth.
the next day or two. Your loyalty points will be credited once this is presented as a secondary query ● Highlight Key Information: reasoning emphasizes all the
the order is finalized. rather than the main request. crucial details that inform the decision.
User: ok bet, just making sure. Customs realy be moving 3. The user does not request help with ● In Depth Analysis of Input: the analysis is in-depth, no
slow fr fr tracking or refund. superficial points
Agent: I understand the frustration. Customs delays have 4. Any ambiguity between a delivery-related ● Free of Unrelated or Repetitive Points: all points must
affected multiple shipments this week, but yours should
intent and a loyalty-related intent is serve to proving the ground truth, avoid repetitive and
clear soon. Your loyalty benefits remain fully intact irrelevant information.
resolved by the fact that the user’s input
● Explain Any Ambiguities - reasoning address any
focuses on when loyalty points are awarded,
Step 4 | Input not the delivery issue brought up in the
uncertainties in the input/context and explain how they are
resolved: delay vs loyalty program intent.
earlier communication. ● Avoid Patterns: no repeated phrasing
gotchu, thanks! and just to dbl check, when you say
loyalty points will be credited once the order is finalized 5. The primary user's intent is ● Avoid "We" & "I" Pronouns: reasoning trace is objective
you mean when the pens arrive? or when the customs Loyalty_Program_Question. and impersonal.
clears?
WRITING REASONING
TRACE Example Page # Page 15 15
Use of any LLM or AI tools is strictly
prohibited for all parts of this task,
including the Input, Context, and
Reasoning Trace. All content must be
written entirely by you.
Page # Page 16 16