wide-ranging review manipulation...

Department of Computer Science and EngineeringTexas A&M University

CIKM 2019

Parisa Kaghazgaran, Majid Alfifi, James Caverlee

Wide-Ranging Review Manipulation Attacks:

Model, Empirical Study and Countermeasure

User Reviews are Everywhere

Online Retailers

Business Review Forums

… and so are targets of manipulation.

Media Platforms

!2

Amazon Headphones, 2019

!3

Amazon Headphones, 2019

!4

Crowd-based Manipulation Campaigns

Crowdsourcing websites

✓Read the product description before writing down a review.

✓Go to https://goo.gl/7QfW0h✓Leave a relevant 5-star review with at

least 40 words.✓Provide the proof that you left the

review your self.

Target Review Platforms

!5

Realistic Reviews: Written by Humans

• Reviews arrive synchronized in time.e.g., Mukherjee et al. 2013, Akoglu et al. 2015, Kaghazgaran et al. 2019

• Dense community over co-review graph.e.g., Kijung et al. Shin 2017, Kaghazgaran et al. 2018

Typical detection approaches:

Manipulators can launch scalable and difficult to detect attacks by removing manipulation traces.

!6

Crowd vs. Machine

!7

Review PlatformsCrowdsourcing

websites

Crowdworkers

Attack Strengths• Scalability: do not rely on paying workers.• Increased deception: obfuscate signals left by crowd campaigns.

!8

How machine produces human readable reviews?

Seminal work Yao, Yuanshun, et al. "Automated crowdturfing attacks and defenses in online review systems." CCS, ACM, 2017.

• Domain dependent reviews.e.i., training process needs to be replicated for any domain, needs large training data for each domain

• Character-level language model.e.i., capture longer dependency, learn spelling in addition to semantics, grammatically more error-prone

Leveraging neural language models for the specific domain of restaurant reviews at Yelp

Our Proposed DIOR Framework

1.WeproposeDIORforDomainIndependentOnlineReviewGenera9on.

2.Empiricalstudytoevaluatethequalityofmachine-generatedreviews.

3.Embedding-basedclassifiertodetectsuchfakereviews.

!9

Goal:validatethewide-ranginga2acksonreviewpla7ormsandproposetodetectthem.

!10

Neural Language Models (A quick refresher)

• Recurrent Neural Networks have shown success in generating meaningful text.

• They learn from a sequence of words to predict the next word

Ht

ot

xt

Each word in the review as input

Learn the information from sequence until time-step t

Predict the next word in the review

x( < = t)(P(xt+1 |x1, …, xt)) H0

o0

x0

H1

o1

x1

H2

o2

x2

H3

o3

x3

I

ate

ate

at this restaurant

At thisIn generation step, the predicted word at time-step t is feed back to the model as input along with hidden state to predict the next word

!11

RQ1: Can we generate reviews across different domains?

I ’ve eaten here about 8 times . I ’ve been introduced to this place . Its always busy and their food is consistently great . I LOVE their food , hence the name . It is so clean , the staff is so friendly , and the food is great . I especially like the chicken pad thai , volcano roll , and the yellow curry .

the case works great ! it has a soft rubber insert that goes over the hard shell . The hard plastic shell has a soft inner shell and the hard case is hard plastic . It is very sticky and has not fallen out or dropped or fallen apart .

this app is a great tool for discovering new things : being able to search for films and putting reviews on particular items as well as having a way to download stories from the app .

!12

Yelp

LSTM3

wt

Encoder

word to id

Input word

id

LSTM2

Decoder

id to word

wt+1

Next word

LSTM1id

Embedding of input word (400)

ht1(1150)

ht2(1150)

ht3(400)Embedding of output word

Transfer Learning to the Rescue!

Amazon

Universal Model

Transferred Model

App Store

Universal model parameters —>

Transferred model parameters —>

θYelp

θAmazon

θAppStore

!13

Example of Synthetic Reviews

I ’ve eaten here about 8 times . I ’ve been introduced to this place . Its always busy and their food is consistently great . I LOVE their food , hence the name . It is so clean , the staff is so friendly , and the food is great . I especially like the chicken pad thai , volcano roll , and the yellow curry .

this is a nice case . It ’s a little difficult to remove , but that ’s to be expected . The case is slightly thicker than a regular screen protector , but that is to be expected . It ’s a great phone case and I highly recommend it .

this app is great for learning the basics of math ! I love that it has a different function that can help you learn the words that you understand . I wish all apps were this simple .

Temperature

Labe

led

“Rea

l” (%

)

020406080

0.2 0.4 0.6 0.8 1.0

Labe

led

“Rea

l” (%

)

022.5

4567.5

90

0.2 0.4 0.6 0.8 1.0

Labe

led

“Rea

l” (%

)

020406080

0.2 0.4 0.6 0.8 1.0

Yelp

Amazon

App Store

!14

RQ2: Can Model-generated Reviews Pass Human Test?

Takeaway 1: Reviews generated at temperature 0.8 can fool human readers and go undetected.Takeaway 2: Human readers are more sensitive to repetition errors than they are to small grammar mistakes.

AMT Guidelines

• 95% approval rate

• Dwell time >= 7 minutes

• Located in US

• Ask for a trivial question

!15

RQ3: Can Spam Detector Catch Model-generated Reviews?

Takeaway: The textual-based spam detector does not distinguish synthetic reviews from real reviews.

Yelp Amazon App Store

Accuracy (%) 64 61 62

Precision (%) 65 64 62

Recall (%) 65 61 62

F1 score (%) 65 60 62

Textual Features

• Similarity

• Structural

• Syntactic

• Semantic

!16

RQ4: How DIOR Works versus Crowd Manipulators?

Takeaway: Users find reviews generated by DIOR as reliable as fake reviews written by manipulation campaigns.

DIOR31%

Neither37%

Crowd32%

!17

RQ5: How DIOR works versus Individual Models?

Takeaway: Using transfer learning not only facilitate the domain shift but also improves the performance significantly.

Pref

eren

ce (%

)

0

25

50

75

100

0

25

50

75

100Amazon App Store

TransferredModel Transferred

Model

IndividualModel

IndividualModelBoth Both

!18

RQ6: How Much Training Data for Transferred Model?

Takeaway: The transferred models need reasonably low number of samples compared to universal model to reach stable performance.

Valid

atio

n Lo

ss

3

3.2

3.4

3.6

25k 50k 75k 100k 125k 150k 200k

App Store Amazon

Training Size

!19

RQ7: How We Can Detect Model-generated Reviews?

Takeaway: Model-generated reviews are detectable in the embedding space with high accuracy.

• Embedding based Classifier

Conclusion and Next Steps

!20

Explored how transferred learning technology could lead to a wide-ranging review manipulation attacks.

Proposed DIOR framework demonstrates:

(1) Model-generated reviews can be perceived as real by human examiners, pass the traditional textual-based spam detectors, and beat the crowd-based review manipulators.

(2) Fake reviews tend to cluster together in the embedding space that provide the intuition for our proposed discriminator.

Next steps: study the performance of other neural network architectures to develop more powerful discriminator.

kaghazgaran@tamu.edu

http://people.tamu.edu/~kaghazgaran/