Developing a Content-Based Spam Detection Method

Abstract

The dramatically increasing number of email users, and the increasing number of free email providers, like yahoo, hotmail, gmail, increase the number of unwanted emails which is known as 'Spam emails'. The huge number of spam emails received daily by users account, made the necessity of existence of some sort of automated spam filters to detect and remove these unwanted emails. Several researchers have started working on automated techniques and tools that can be used to classify emails automatically into wanted) legitimate) or unwanted (spam) emails. Most of these filters are based on naïve Bayesian method. This thesis introduces a new automated filter based on naïve Bayesian. The basic idea of this filter is to give each word appears in emails a probabilistic value, this value indicates its probable belonging to spam. As there are many common words appear in spam as well as legitimate messages with the same rate, the proposed filter has a preprocessing component which removes all common words. The researcher carefully collected these common words. In the training phase a set of 1300 emails (legitimate and Spam) has been used. In this phase the weight of every uncommon word is estimated as the probability of a given word in spam email divided by the probability of the same word in legitimate email. In classification, a Bayesian classifier uses the weight table generated in the training phase to classify a given email as spam or legitimate. The proposed filter has been tested on a dataset of 400 emails, 200 of them are Spam and 200 of them are legitimate, the proposed algorithm succeeded in detecting 90% of the spam messages.