A distributed joint sentiment and topic modeling using spark for big opinion mining

Zahedi, E; Saraee, MH; Baniasadi, Z

Abstract

Opinion data are produced rapidly by a large and uncontrolled number of opinion holders in different domains (public, business, politic and etc). The volume, variety and velocity of such data requires an opinion mining model to be also adopted with the ever growing and huge volume of opinions and obtaining the probabilistic generative model advantages. In this paper we propose a parallel implementation of joint sentiment and topic (JST) model for simultaneously discovering topics and sentiments from reviews on Spark. Spark is an open source and fast cluster computing framework for large-scale data processing. Here we discuss the implementation of JST on Spark and also discuss the benefit of using Spark while exploring the challenges encountered. We used different Amazon opinion datasets with different volume such as (reviews of electronic devices, book, restaurants, DVD and kitchen). The results present significant speedup and high efficiency on larger scale dataset in our experiments. Index Terms—big opinion dataset, joint sentiment and topic model, Spark, cluster computing

Citation

Zahedi, E., Saraee, M., & Baniasadi, Z. (2017). A distributed joint sentiment and topic modeling using spark for big opinion mining. In Iranian Conference on Electrical Engineering (ICEE), 2017 (1475-1480). IEEE. https://doi.org/10.1109/IranianCEE.2017.7985276

• Start Date: May 2, 2017
• End Date: May 4, 2017
• Online Publication Date: Jul 20, 2017
• Publication Date: Jul 20, 2017
• Deposit Date: Jul 10, 2017
• Pages: 1475-1480
• Book Title: Iranian Conference on Electrical Engineering (ICEE), 2017
• ISBN: 9781509059638
• DOI: https://doi.org/10.1109/IranianCEE.2017.7985276
• Publisher URL: http://dx.doi.org/10.1109/IranianCEE.2017.7985276
• Related Public URLs: http://icee2017.kntu.ac.ir/en/
• Additional Information: Event Type : Conference