does presidential debates have impact on election results?

Active Learning Project 1

Does the Presidential debates have any impact on election results?

Ashwath VAvudaiappan R

Harish ASudhakaran S

Suganthy E

Monday, January 18, 2016


Big Data (Hadoop - Map Reduce, Pig, Hive) – for input text file processing R & Tableau – for Visualization Excel – for output interpretation

Tools and Technologies used

Referencehttp://www.debates.org/index.php?page=debate-transcripts


http://www.debates.org/index.php?page=debate-transcripts

3

Does the Presidential debates have any impact on election results?Monday, January 18, 2016

Does the Presidential debates have any impact on election results?

Objective:• Finding out the word frequency and distribution for different parts of speech.• Categorization of Positive and Negative words from the speech and if it has any impact

on election results

Our analysis will be helpful, If you are any of the following:• Do you want to contest in the next Presidential Elections?• Would you like to know which of the two candidates fared well after the debate?• Are you a political or a media person?

Sources:www.debates.org

Well, my first job as commander in chief, is to keep

the American people safe.

We need strong leadership. I'd like to be that leader with your

support. I'll work with you.

Monday, January 18, 2016 Active Learning Project 4


MAPPER -Code to find word count

/**The mapper reads one line at the time, splits it into an array of single words and emits every word to the reducers with the value of 1**/

public static class TopNMapper extends Mapper<Object, Text, Text, IntWritable> {

private final static IntWritable one = new IntWritable(1); private Text word = new Text(); private String tokens = "[_|$#<>\\^=\\[\\]\\*/\\\\,;,.\\-:()?!\"']";

@Override public void map(Object key, Text value, Context context) throws IOException, InterruptedException { String cleanLine = value.toString().toLowerCase().replaceAll(tokens, " "); StringTokenizer itr = new StringTokenizer(cleanLine); while (itr.hasMoreTokens()) { word.set(itr.nextToken().trim()); context.write(word, one); }}}

MAP-REDUCE



REDUCER -Code to find word count

/** The reducer retrieves every word and puts it into a Map: if the word already exists in the map, increments its value, otherwise sets it to 1**/ public static class TopNReducer extends Reducer<Text, IntWritable, Text, IntWritable> {

private Map<Text, IntWritable> countMap = new HashMap<>(); @Override public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {

// **computes the number of occurrences of a single word**/ int sum = 0; for (IntWritable val : values) { sum += val.get();}



OUTPUT


Code to find word count

/**Loading data**/debate_dataset = LOAD '/home/cloudera/Desktop/debate.txt' AS (line:chararray);

/** Extract words from each line and put them into a pig bag**/ /** datatype, then flatten the bag to get one word on each row**/ words = FOREACH input_lines GENERATE FLATTEN(TOKENIZE(line)) AS word; /**filter out any words that are just white spaces**/ filtered_words = FILTER words BY word MATCHES '\\w+'; /**create a group for each word**/ word_groups = GROUP filtered_words BY word; /**count the entries in each group**/ word_count = FOREACH word_groups GENERATE COUNT(filtered_words) AS count, group AS word; /**order the records by count**/ ordered_word_count = ORDER word_count BY count DESC; STORE ordered_word_count INTO '/home/cloudera/Desktop/wordcountoutput’;

PIG



OUTPUT



I wonder who spoke more positively?? How to find it??

Hadoop Map

Reduce

VS



Positive words Negative words


Map Reduce codepublic class CompareTwoFiles { public static class Map extends Mapper<LongWritable, Text, LongWritable, Text> { @Override public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { context.write(key, value); }} public static class Map2 extends Mapper<LongWritable, Text, LongWritable, Text> { @Override public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { context.write(key, value); }}public static class Reduce extends Reducer<LongWritable, Text, LongWritable, Text> { @Override public void reduce(LongWritable key, Iterable<Text> values, Context context) throws IOException, InterruptedException



Map Reduce code{ String[] lines = new String[2]; int i = 0; for (Text text : values) { lines[i] = text.toString(); i++; } if (lines[0].equals(lines[1])) { context.write(key, new Text("same")); } else { context.write(key, new Text(lines[0] + " vs " + lines[1])); } } } public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); conf.set("fs.default.name", "hdfs://localhost:8020"); Job job = new Job(conf); job.setJarByClass(CompareTwoFiles.class); job.setJobName("Compare Two Files and Identify the Difference"); FileOutputFormat.setOutputPath(job, new Path(args[2])); job.setReducerClass(Reduce.class); job.setOutputKeyClass(LongWritable.class); job.setOutputValueClass(Text.class); MultipleInputs.addInputPath(job, new Path(args[0]), TextInputFormat.class, Map.class); MultipleInputs.addInputPath(job, new Path(args[1]), TextInputFormat.class, Map2.class); job.waitForCompletion(true); } }


Positive words Output



Obama Romney0

50

100

150

200

250

194

153

Obama VS Romney Po

sitive

wor

d c

ount

Final Interpretation

Our Analysis reveals that positive words spoken by Barack Obama is comparatively higher than Mitt Romney. Coincidentally Obama was elected as the President of United States of

America.Yes, debates do have an impact on the Election results !



Positive words cloud



THANK YOU


does presidential debates have impact on election results?

Data & Analytics