SUR-Miner Research Project

Appendix

1. Typo list

2. Templates

3. Detailed Results

Typo List

Typo Replaced With
" im "," i m ","I m " " I'm "
"Ill " "I will "
" cant "," cnt " " can't "
"doesnt" "doesn't"
"dont""don't"
"isnt" "isn't"
"didnt" "didn't"
"couldnt" "couldn't"
"Cud ""Could "
"wasnt" "wasn't"
"wont" "won't"
"wouldnt" "wouldn't"
" ive " " I have "
"Ive ""I have "
" theres " " there's "
"awsome", "awsum","awsm ""awesome"
"Its " "It's "
" dis "," diz " " this "
"plzz","Plz ","plz","pls ","Pls","Pls ""please"
" U "," u " " you "
" ur " " your "
" b " " be "
" r " " are "
" nd "," n "," & " " and "
" bt " " but "
" nt " " not "
" coz "," cuz " " because "
"jus "," jst " "just "
"luv","Luv" "love"
"gud" "good"
"Gud" "Good"
" wel " " well "
"gr8","Gr8","Grt "," grt " "great"
"Gr\\." "Great\\."
"pics " "pictures
"pic " "picture "
" hav " " have "
" nic " " nice "
"nyc ","9ce" "nice "
"Nic ""Nice "
" agn " " again "
"thx","tks ","thanx" "thanks"
"Thx" "Thanks"
"thkq""thank you"
" wel " " well "
"Xcellent" "excellent"
" vry " " very "
"Vry " "Very "
" fav "," favo " " favorite "
" soo " " so "
"sooo","soooo" "so"
" ap " " app "
" b4 " " before "
" ez " " easy "
" w8 " " wait "
"msg" "message"
"alot" "a lot"
" lota " " lot of "
" kinda " " kind of "
"omg" "oh, my god"
" gota " " going to "

Templates

Templates Sample Aspect Words Opinion Words
JJ[nsubj-NN,cop-VBZ]   nsubj-NN JJ
JJ[nsubj-NN[nn-NN],cop-VBZ,neg-RB]   nn-NN + nsubj-NN neg-RB + JJ
JJ[nsubj-NN[nn-NN],cop-VBZ,rcmod] glad 'it has arabic support' rcmod JJ
JJ[nsubj-NN[nn-NN],cop-VBZ,dep]   dep JJ
JJ[nsubj-NN[nn-NN],cop-VBZ,npadvmod] 'Prediction' better than.. npadvmod JJ
JJ[nsubj(I),prep(with..),..] I am frastrated with the..   "I am" +JJ+ "with"
NN[amod-JJ] beautiful UI NN amod - JJ
VB(have)[nsubj-NN,nobj-NN]   nsubj-NN have + nobj-NN
VB(is|are|seems..)[nsubj,obj] The button is elegant. subj obj
VB(is|are|seems..)[nsubj,prep-JJ] The button is as good as.. nsubj prep
VB(is|are|seems..)[nsubj,dep]      
VB(is|are|seems..)[nsubj,advmod]   nsubj advmod
VB(like|enjoy..)[nsubj,nobj-NN] I like the button. nobj-NN like|enjoy..
VB(like|enjoy..)[nsubj,acomp]   acomp like|enjoy..
VB(like|enjoy..)[nsubj,ccomps] I don't like that there is a big screen    
VB[expl,nsubj]   nsubj like|enjoy..
VB[expl,nobj]   nobj like|enjoy..
VB(like|enjoy..)[xcomp] don't like 'to..' xcomp like|enjoy..
VB(like|enjoy..)[prep]   prep like|enjoy..
VB[nsubj,nobj] It drains my battery nsubj nobj
VB[nsubj,advmod] It predicts very well nsubj advmod
VB[nsubj,acomp] It works nearly perfect nsubj acomp
VB[nsubj,xcomp]   nsubj xcomp
NN(love|hate|enjoy..)[xcomp] love 'the text to email' xcomp love|hate|enjoy..
NN(love|hate|enjoy..)[dep]   dep love|hate|enjoy..
NN[nsubj,cop] the new button is a good update nsubj cop+NN

Detailed Results

Precisions and Recalls for Review Classification
Subject Evaluation Praise Request Bug Other
P R P R P R P R P R
Swiftkey 0.75 0.70 0.86 0.88 0.82 0.73 0.76 0.46 0.81 0.90
WeChat 0.79 0.65 0.92 0.93 0.78 0.37 0.89 0.68 0.85 0.94
Camera360 0.78 0.67 0.93 0.97 0.79 0.30 0.89 0.70 0.82 0.88
Templerun2 0.78 0.75 0.83 0.86 0.84 0.54 0.88 0.78 0.73 0.82
GoogleMap 0.77 0.70 0.87 0.90 0.76 0.51 0.80 0.73 0.80 0.87
KakaoTalk 0.82 0.70 0.93 0.97 0.81 0.57 0.81 0.32 0.88 0.95
YahooWeather 0.85 0.83 0.92 0.95 0.77 0.47 0.84 0.40 0.83 0.90
SpotifyMusic 0.71 0.64 0.92 0.95 0.72 0.44 0.87 0.46 0.84 0.91
GoogleplayBooks 0.66 0.53 0.90 0.93 0.77 0.68 0.70 0.54 0.82 0.90
Duolingo 0.84 0.76 0.94 0.96 0.81 0.58 0.60 0.13 0.85 0.92
Text+ 0.72 0.60 0.93 0.96 0.74 0.30 0.83 0.65 0.84 0.93
ESPN 0.81 0.74 0.85 0.76 0.72 0.48 0.80 0.74 0.79 0.89
GoogleCalendar 0.76 0.72 0.80 0.74 0.81 0.79 0.88 0.60 0.78 0.85
ChaseMobile 0.82 0.70 0.94 0.92 0.62 0.41 0.69 0.49 0.80 0.90
Medscape 0.85 0.83 0.94 0.95 0.75 0.55 0.93 0.59 0.85 0.91
Yelp 0.80 0.75 0.93 0.90 0.80 0.28 0.94 0.45 0.87 0.94
IMDB 0.76 0.66 0.88 0.92 0.70 0.52 0.78 0.58 0.80 0.89

 

Precisions and Recalls for Aspect-Opinion Extraction
Subject Aspect Opinion Sentiment Postive Sentiment Negative
P R P R P R P R
 Swiftkey  0.86  0.88  0.84  0.89  0.97  0.79  0.59  0.87
 Wechat  0.86  0.81  0.85  0.81  0.94  0.65  0.85  0.81
 Camera360  0.86  0.88  0.86  0.87  1.0  0.80  0.41  0.75
 Templerun2  0.95  0.95  0.93  0.93  0.96  0.87  0.75  0.84
 GoogleMap  0.89  0.83  0.87  0.81  0.87  0.89  0.97  0.81
 KakaoTalk  0.84  0.84  0.90  0.83  0.95  0.76  0.68  0.87
 Yahooweather  0.91  0.89  0.89  0.89  1.0  0.80  0.71  0.84
 Spotifymusic  0.83  0.86  0.85  0.87  0.95  0.74  0.64  0.60
 Googleplaybooks  0.82  0.86  0.85  0.87  0.93  0.73  0.86  0.78
 Duolingo  0.85  0.87  0.84  0.86  0.99  0.87  0.36  0.91
 Text+  0.81  0.87  0.84  0.87  0.92  0.70  0.75  0.79
 Espn  0.78  0.81  0.77  0.80  0.83  0.59  0.84  0.82
 GoogleCalendar  0.75  0.84  0.79  0.85  0.90  0.69  0.87  0.83
 ChaseMobile  0.80  0.89  0.86  0.90  0.94  0.81  0.80  0.75
 Medscape  0.88  0.91  0.90  0.90  1.00  0.75  0.44  0.92
 Yelp  0.83  0.86  0.88  0.86  0.99  0.81  0.76  0.89
 IMDB  0.82  0.86  0.87  0.88  0.94  0.79  0.83  0.79