Using web scraping in social media to determine market trends with product feature - based sentiment analysis

Abstract

Customer product reviews are openly available online and they are now widely used for deciding quality of product or service and to determine market trends and influence decision making of users. Due to the availability of a massive number of customer reviews on the web, summarizing them requires a fast classification system. Compared to supervised and unsupervised machine learning techniques for binary classification of reviews, fuzzy logic can provide a simple and comparatively faster way to model the fuzziness existing between the sentiment polarities classes due to the uncertainty present in most of the natural languages. But the fuzzy logic techniques are not much considered in this domain. This thesis proposes a model which measures product market value by using sentiment analysis conducted on the reviews of online products which are collected from a well known ecommerce website “Amazon”. Fuzzy logic approach is used in calculating the final product market demand. Hence, in this paper we propose a fine grained classification of customer reviews into weak positive, average positive, strong positive, weak negative, average negative and strong negative classes using a fuzzy logic model based on the most popularly known sentiment based lexicon SentiWordNet. By creating rules and relationships between fuzzy membership functions and linguistic variables, we can analyze the customer opinions towards online products. This proposed model provides the most reasonable sentiment analysis because we try to reduce all the problems from the related past researches. The outcomes can allow the business organization to understand their customer‟s sentiments and improve customer loyalty and customer retention techniques in order to increase customer values and profits result. Fine grained classification accuracy approximately in the range of 74% to 77% has been obtained by experiments conducted on datasets of electronic products containing reviews of smart phones, TV and laptops.