policy change indexpolicychangeindex.org/pdf/zhong_gec_tech_demo_slides.pdfpolicy change index...
TRANSCRIPT
![Page 1: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/1.jpg)
Policy Change Index
AI-Assisted Propaganda Analysis
Weifeng Zhong, Ph.D.
Senior Research Fellow
Mercatus Center at George Mason University
January 29, 2020
GEC Tech Demo
policychangeindex.org
![Page 2: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/2.jpg)
Policy Change Index (PCI) project
• Predict authoritarian policies based on propaganda;
• First use cases: China;
• Future: North Korea, Iran, Cuba, Venezuela, Russia,...
• Open-source and free!
policychangeindex.org
![Page 3: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/3.jpg)
PCI has been demonstrated to
• White House OSTP, USTR;
• DoD Project Maven, Naval Research Lab, In-Q-Tel;
• Bipartisan members of Congress, including House
Ways & Means, House Foreign Affairs, and House
Armed Services;
• Federal Reserve Board.
policychangeindex.org
![Page 5: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/5.jpg)
“Value” of closed regimes’ propaganda
High, because
policychangeindex.org
propaganda content policy intentions𝒊𝒏𝒅𝒊𝒄𝒂𝒕𝒆𝒔
![Page 6: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/6.jpg)
“Value” of closed regimes’ propaganda
High, because
policychangeindex.org
propaganda content policy intentions𝒊𝒏𝒅𝒊𝒄𝒂𝒕𝒆𝒔
![Page 7: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/7.jpg)
Lessons from history
• The Soviets;
policychangeindex.org
(Lenin and Stalin, 1922)
![Page 8: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/8.jpg)
Lessons from history
• The Soviets;
• The Nazis;
policychangeindex.org
(Alexander George, 1959)
![Page 9: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/9.jpg)
Lessons from history
• The Soviets;
• The Nazis;
• The Chinese;
policychangeindex.org
(Xi Jinping at CCTV, 2016)
![Page 10: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/10.jpg)
Lessons from history
• The Soviets;
• The Nazis;
• The Chinese;
• And many more.
policychangeindex.org
![Page 11: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/11.jpg)
Analyzing closed regimes’ propaganda
Easier said than done!
• Unstructured data;
• Labor-intensive;
• Objective mixed with subjective.
policychangeindex.org
![Page 12: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/12.jpg)
Dr. Whatson’s headache
• “Dr. Whatson” is monitoring “unusual” content in the
People’s Daily,
• and spots an “unusual” article on the front page.
policychangeindex.org
![Page 13: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/13.jpg)
Dr. Whatson’s headache
• “Dr. Whatson” is monitoring “unusual” content in the
People’s Daily,
• and spots an “unusual” article on the front page.
• Is it really unusual?
policychangeindex.org
3 months’ worth5 years’ worth
![Page 14: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/14.jpg)
Dr. Whatson’s headache
• “Dr. Whatson” is monitoring “unusual” content in the
People’s Daily,
• and spots an “unusual” article on the front page;
• Is it really unusual?
• A question for every day!
• How to evaluate Dr. Whatson?
policychangeindex.org
![Page 16: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/16.jpg)
Core idea
policychangeindex.org
AI can help!
propaganda content policy intentions𝒊𝒏𝒅𝒊𝒄𝒂𝒕𝒆𝒔
![Page 17: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/17.jpg)
Key features
• Automate content analysis of propaganda;
• Delineate boundaries between objective and
subjective components.
policychangeindex.org
![Page 19: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/19.jpg)
PCI-China
• Input: People’s Daily full text, 1946-present;
• Output: quarterly index of major policy changes.
policychangeindex.org
![Page 20: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/20.jpg)
PCI-China: algorithm
policychangeindex.org
PCI-China: difference in performance
Articles in next
quarter
Deploy
Articles in previous 5 years
Train a front-page classifier
appx
![Page 21: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/21.jpg)
PCI-China: result
(1951 Q1 to 2019 Q4)
appx
![Page 22: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/22.jpg)
PCI-China predicts major changes
(1951 Q1 to 2019 Q4)
appx
![Page 23: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/23.jpg)
PCI-China predicts major changes
(1951 Q1 to 2019 Q4)
appx
![Page 24: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/24.jpg)
Machine’s mistakes are meaningful
policychangeindex.org
Classified on front page?
No Yes
Front page?No False positives
Yes False negatives
New talking points Old talking points
appx
![Page 25: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/25.jpg)
From propaganda to policy
policychangeindex.org
Classified on front page?
No Yes
Front page?No False positives
Yes False negatives
New talking points Old talking points
New policies Old policies
Contains
judgment
appx
![Page 26: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/26.jpg)
PCI-China: what’s the latest?
(1951 Q1 to 2019 Q4)
2018 Q1
2019 Q1
appx
![Page 27: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/27.jpg)
E.g., 2018 Q1 spikeContains
judgment
appx
![Page 28: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/28.jpg)
US-China trade implications
• Two spikes in 2018 Q1 and 2019 Q1:
• Internal: strengthening party authority;
• External: nationalism and global leadership.
• Our prediction since 2019 Q1: curb your enthusiasm.
• 2019 Q4: still no signs of serious changes.
Contains
judgment
policychangeindex.org
appx
![Page 29: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/29.jpg)
Dr. Whatson would sayContains
judgment
Hard-line Pro-market
propaganda content policy intentions𝒊𝒏𝒅𝒊𝒄𝒂𝒕𝒆𝒔
policychangeindex.org
![Page 30: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/30.jpg)
What PCI did for Dr. Whatson
• Automate the detection of “unusual” content;
• Delineate boundaries between the “unusual” and how
Dr. Whatson interprets it.
policychangeindex.org
![Page 31: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/31.jpg)
Other closed regimes?
• PCI-NK;
• PCI-Iran;
• PCI-Cuba;
• PCI-Venezuela;
• PCI-Russia (and PCI-USSR);...
policychangeindex.org
![Page 32: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/32.jpg)
Can’t wait a quarter?
E.g., will the Hong Kong protests be met with
a Tiananmen-like crackdown? If so, when??
policychangeindex.org
![Page 34: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/34.jpg)
PCI-Crackdown
• Input: People’s Daily articles on Tiananmen and
Hong Kong protests;
• Output: daily index of crackdown possibility.
policychangeindex.org
![Page 35: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/35.jpg)
PCI-Crackdown: algorithm
policychangeindex.org
PCI-Crackdown: “as-if” date for the actual date
Recent articles
on HK protests
Calculate the
“as-if” date
Articles leading up to
Tiananmen crackdown
Train a date classifier
appx
![Page 36: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/36.jpg)
PCI-Crackdown monitors HK protests
(Jun 9, 2019 to Jan 7, 2020)
appx
![Page 37: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/37.jpg)
Dr. Whatson would sayContains
judgment
Fluctuations in Fluctuations in
propaganda content policy intentions𝒊𝒏𝒅𝒊𝒄𝒂𝒕𝒆𝒔
policychangeindex.org
![Page 38: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/38.jpg)
Where’s the judgment call?
• There is a well-formed precedent;
• The same “playbook” is being followed now.
policychangeindex.org
Contains
judgment
![Page 39: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/39.jpg)
Other timeline mappings?
• Wuhan coronavirus (vis-a-vis SARS)?
• China’s Taiwan policy?
• NK missile tests?
• Other rare events?
policychangeindex.org
![Page 41: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/41.jpg)
Core idea
policychangeindex.org
AI can help!
propaganda content policy intentions𝒊𝒏𝒅𝒊𝒄𝒂𝒕𝒆𝒔
![Page 42: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/42.jpg)
How can PCI be useful to you?
• First step toward policymaking is to better understand
the problem.
• PCI’s mission: crack a window to otherwise closed
systems with new techniques.
• You should start peeking inside — for free!
policychangeindex.org
![Page 43: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/43.jpg)
Thank you!
• Website: policychangeindex.org
• Newsletter: mercatus.org/PCInewsletter
• Code: github.com/PSLmodels/PCI-China
github.com/PSLmodels/PCI-Crackdown
• Developers:
• Julian Chan (Bates White), [email protected]
• Weifeng Zhong (Mercatus), [email protected]
policychangeindex.org
![Page 45: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/45.jpg)
PCI-China: model
Texts
Word
embedding
Recurrent
neural
networks
Metadata
Multilayer
perceptron
Multilayer perceptron
Front page?
Input
Neural
networks
Output
𝒙 : each article as an obs.
𝒚 = 𝒇 𝒙
𝒇 : a complicated function
back
![Page 46: Policy Change Indexpolicychangeindex.org/pdf/Zhong_GEC_Tech_Demo_Slides.pdfPolicy Change Index AI-Assisted Propaganda Analysis Weifeng Zhong, Ph.D. Senior Research Fellow Mercatus](https://reader033.vdocument.in/reader033/viewer/2022050503/5f95a0506010855cd05b99a7/html5/thumbnails/46.jpg)
PCI-Crackdown: model
Texts
Word
embedding
Recurrent
neural
networks
Multilayer perceptron
Date?
Input
Neural
networks
Output
𝒙 : each sentence as an obs.
𝒚 = 𝒇 𝒙
𝒇 : a complicated function
back