Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Best of 2016 Recommended by Editor Recommended by Reviewer Recommended by Reader
Search
Sentiment analysis for software engineering: How far can we go?
Lin B., Zampetti F., Bavota G., Di Penta M., Lanza M., Oliveto R.  ICSE 2018 (Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden, May 27-Jun 3, 2018)94-104.2018.Type:Proceedings
Date Reviewed: Aug 15 2019

Sentiment analysis continues to be successfully applied to consumer reviews. In other areas, the challenges involved can prove insurmountable. A failed attempt to successfully apply sentiment analysis is reported in this negative results paper.

The investigators designed a system that would have been capable of recommending software libraries based on text extracted from Stack Overflow discussions. They adopted a “state-of-the-art approach based on a recursive neural network,” Stanford CoreNLP, to analyze the extracted text for sentiments. A training set involving the manual labeling of sentiments required some 90 hours of work to build. Despite this best practice effort, as the final row of table 2 indicates, overall “precision and recall in detecting positive and negative sentiments [was less than] 40 percent.”

As a follow up, the investigators evaluated five sentiment analysis tools on three datasets (Stack Overflow discussions, app reviews, and JIRA issues). Results for the Stack Overflow discussions dataset were, as before, not acceptable. For the app reviews dataset, however, results were acceptable. This success was attributed to the fact that the reviews were similar to consumer reviews in which opinions are clearly expressed. Results for the JIRA issues dataset were also acceptable, but the interpretation is complicated by the fact that there are no neutral sentences in this particular dataset. The investigators come to the inevitable conclusion that opinion mining a dataset comprising developer discussions of technicalities is obviously a very difficult challenge.

This paper provides several useful insights and is strongly recommended to those working on sentiment analysis.

Reviewer:  Andy Brooks Review #: CR146654 (1911-0393)
Bookmark and Share
  Reviewer Selected
Editor Recommended
Featured Reviewer
 
 
General (D.2.0 )
 
 
Natural Language Processing (I.2.7 )
 
Would you recommend this review?
yes
no
Other reviews under "General": Date
Development of distributed software
Shatz S. (ed), Macmillan Publishing Co., Inc., Indianapolis, IN, 1993. Type: Book (9780024096111)
Aug 1 1994
Fundamentals of software engineering
Ghezzi C., Jazayeri M., Mandrioli D., Prentice-Hall, Inc., Upper Saddle River, NJ, 1991. Type: Book (013820432)
Jul 1 1992
Software engineering
Sodhi J., TAB Books, Blue Ridge Summit, PA, 1991. Type: Book (9780830633425)
Feb 1 1992
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy