UNIT-3
Topics to be covered
• Machine learning Algorithms and Applications –
• Motivating application: Filtering Spam –
• Why Linear Regression and k-NN are poor choices for Filtering Spam
• Naive Bayes and why it works for Filtering Spam
•
Machine learning Algorithms and
Applications
What is a spam filter?
• Spam filters are designed to identify incoming dangerous emails from
attackers or marketers. Attackers often use emails that claim to offer
a beneficial service or protect you from imminent danger, but they
are really just clickbait, designed to get you to click on a link that
downloads malicious software onto your computer or sends you to a
dangerous site.
• Spam can also contain relatively harmless content but can clutter up
your inbox, consuming valuable space and making it more difficult to
identify important, useful emails. Spam filters can detect spam
emails. These helpful tools can recognize patterns that spam emails
tend to follow.
• Because an email spam filtering can recognize these kinds of emails, it
can be a valuable solution for protecting users from unwanted
messages. To enhance the protection, some spam filters use insights
gained from machine learning to more accurately target junk mail.
• Often, an internet service provider (ISP) will use spam detection to
reduce the amount of spam delivered to users. Companies can do the
same to lessen the burden of spam on their employees. However,
spam filtering can also flag legitimate emails from companies users
actually want to get messages from. In this case, users often have the
option to determine the kinds of emails they want and adjust their
settings to allow them to pass through the filter.
How do spam filters work?
• Spam filters use “heuristics” methods, which means that each email message is
subjected to thousands of predefined rules (algorithms). Each rule assigns a numerical
score to the probability of the message being spam, and if the score passes a certain
threshold the email is flagged as spam and blocked from going further.
• There are different types of spam filters for different criteria:
• Content filters – parse the content of messages, scanning for words that are commonly
used in spam emails.
• Header filters – examine the email header source to look for suspicious information
(such as spammer email addresses).
• Blocklist filters – stop emails that come from a blocklist of suspicious IP addresses. Some
filters go further and check the IP reputation of the IP address.
• Rules-based filters – apply customized rules designed by the organization to exclude
emails from specific senders, or emails containing specific words in their subject line or
body.
• You can also run your message through certain tools, such as
Mail Tester to test the ‘spammyness’ of an email.
• No single method is a complete solution to the spam problem, and there are always
trade-offs (which the heuristics try to weigh) between rejecting legitimate email vs.
letting spam slip through.
Why Won’t Linear Regression Work
for Filtering Spam?
• Imagine a dataset or matrix where each row represents a different
email message(it could be keyed by email_id). Now let’s make each
word in the email
• a feature—this means that we create a column called “Viagra,” for
example,and then for any message that has the word Viagra in it at
least once, we put a 1 in; otherwise we assign a 0. Alternatively, we
could put the number of times the word appears. Then each column
represents the appearance of a different word.
• in order to use liner regression, we need to have a training set of
emails where the messages have already been labeled with some
outcome variable.
• In this case, the outcomes are either spam or not. We could do this
by having human evaluators label messages “spam,” which is a
reasonable, but time intensive,solution. Another way to do it would
be to take an existing spam filter such as Gmail’s spam filter and use
those labels.
• Once you build a model, email messages would come in without a
label, and you’d use your model to predict the labels.
• The first thing to consider is that your target is binary (0 if not spam,
• 1 if spam)—you wouldn’t get a 0 or a 1 using linear regression;
• you’d get a number. Strictly speaking, this option really isn’t ideal;
linear regression is aimed at modeling a continuous output and this is
binary.
• Even if we try to use Linear regression we need to filter emails on the
order of 10,000 emails with on the order of 100,000 words but still
the outcome is binary value but linear regression is not the
appropriate model for a binary outcome.
How About k-nearest Neighbors?
• We need to choose features, probably corresponding to words, and
we’d likely define the value of those features to be 0 or 1, depending
on whether the word is present or not.
• Then, we’d need to define when two emails are “near” each other
based on which words they both contain.
• Again, with 10,000 emails and 100,000 wordsNamely, the
• space we’d be working in has too many dimensions.
• Yes, computing distances in a 100,000-dimensional space requires
lots of computational work.
• KNN when used for Email Spam filtering has a problem that even the
nearest neighbors are really far away. This is called “the curse of
dimensionality,” and it makes k-NN a poor algorithm in this case.