A Comparative Studyfor Two Stemming Algorithmsfor Arabic Wikipedia Documents Classification Based on Similarity Measures

Abstract

Text mining is an important field in information retrieval; it organize alargenumber of text documents that available on the internet to facilitate the retrieved processing and increase efficiency. Text classification is automatically determining the category to new or unseen documents that depends on content of document itself. In text classification, text preprocessing is a fundamental step to obtained a better result. The Arabic text processing depends on stemming algorithms to achieve high accuracy. This research aims to compare between two stemming algorithms stem approach (snowball light) and root approach (Shereen Khoja) using three similarity measures: Euclidean distance, cosine similarity, and pearson correlation distance. This research use Arabic Wikipedia dataset and TF-IDF as weight scheme to construct the vector space model to represent the weight of selected features of text. For evaluation measures, the research applies overall accuracy, average recall, average precision, and average F1 measure to assess the results of the classified text documents. The collection of document is divided into training and test documents according to three experimental (85% – 15%) (80% – 20%) (90% – 10%) for training and test document respectively. The results showed the overall accuracy of Shereen Khoja stemmer is better than Snowball stemmerin all experimental excluding cosine similarity in the first experimental and Euclidean distance in the third experimental which has a better accuracy when use Snowball stemmer.