W05-06: Hierarchical Sentiment Analysis on HPV Vaccines Related Tweets Using Machine Learning Based Approaches

In order to figure out the reasons behind the low HPV vaccine coverage and come up with corresponding strategies to improve vaccine uptake, understand public opinions on HPV vaccines would be of great help. As a precious and rich data source to analyze public opinions, Twitter is now attracting more attention from medical informatics researchers. In order to use machine learning based methods to automatically track and analyze rapidly-growing HPV vaccine related public opinions on Twitter, we first collected and manually annotated 6,000 related tweets as a gold standard. In order to model the possible sentiments especially the negative sentiments over HPV vaccine on Twitter, a preliminary ontology for hierarchical sentiment classification was built as the annotation scheme. A Kappa annotation agreement at 0.851 was reached. Different features (word n-grams, POS and word clusters) were extracted. Experiments were conducted to test the performance of the different combinations features sets on different levels of classification tasks. Macro F scores at 0.8043 and 0.7552 were reached for top-level classification and finest level classification respectively. The limitations and challenges were also discussed. Our results and analysis indicate that it is feasible to do hierarchical classification tasks on HPV vaccine related tweets using machine learning approaches.