using clojure for sentiment analysis of the twittersphere (euroclojure 2014)
DESCRIPTION
Slides from "Using Clojure for Sentiment Analysis of the Twittersphere" - EuroClojure 2014. I had to convert from Keynote to pdf for upload, so apologies for loss of animations or notes.TRANSCRIPT
Sentiment Analysis of the Twittersphere
Leiningen Versus the Ants!
Clojure Versus Java!
FP Versus OOP
Leiningen Versus the Ants!
Clojure Versus Java!
FP Versus OOP
Apache Ant
Leiningen Versus the Ants
Clojure Versus Java!
FP Versus OOP
Apache Ant
Leiningen Versus the Ants
Clojure Versus Java
Functional Versus Object Oriented
Apache Ant
Blood aches pain sick shivers spasm vomit dizzy fainting colic and in a coma!
1 FAVORITE
Blood aches pain sick shivers spasm vomit dizzy fainting colic and in a coma!
1 FAVORITE
Scores HIGH for flu symptoms
Blood aches pain sick shivers spasm vomit dizzy fainting colic and in a coma!
1 FAVORITE
Who is Catalina Rubottom?
Blood aches pain sick shivers spasm vomit dizzy fainting colic and in a coma!
1 FAVORITE
HDFS/Hadoop
Mongo/Aggregation
Mongo/MapReduce
Postgres
30 million geo-tagged tweets sent from UK
How can we do fast, real time analytics of
social media?
Store this!
Not this!
ptaoussanis/carmine
(:require [taoensso.carmine :as car]) !!(defn create-tweet-id [] (first (wcar* (car/incr "global:tweet.id")))) !!(defn set-values [sentiment location created-at] (let [tweed-id (create-tweet-id)] (wcar* (car/setbit sentiment tweet-id 1) (car/setbit location tweet-id 1) (car/setbit created-at tweet-id 1))))
(:require [taoensso.carmine :as car]) !!(defn create-tweet-id [] (first (wcar* (car/incr "global:tweet.id")))) !!(defn set-values [sentiment location created-at] (let [tweed-id (create-tweet-id)] (wcar* (car/setbit sentiment tweet-id 1) (car/setbit location tweet-id 1) (car/setbit created-at tweet-id 1))))
(:require [taoensso.carmine :as car]) !!(defn create-tweet-id [] (first (wcar* (car/incr "global:tweet.id")))) !!(defn set-values [sentiment location created-at] (let [tweed-id (create-tweet-id)] (wcar* (car/setbit sentiment tweet-id 1) (car/setbit location tweet-id 1) (car/setbit created-at tweet-id 1))))
(:require [taoensso.carmine :as car]) !!(defn create-tweet-id [] (first (wcar* (car/incr "global:tweet.id")))) !!(defn set-values [sentiment location created-at] (let [tweed-id (create-tweet-id)] (wcar* (car/setbit sentiment tweet-id 1) (car/setbit location tweet-id 1) (car/setbit created-at tweet-id 1))))
(:require [taoensso.carmine :as car]) !!(defn create-tweet-id [] (first (wcar* (car/incr "global:tweet.id")))) !!(defn set-values [sentiment location created-at] (let [tweed-id (create-tweet-id)] (wcar* (car/setbit sentiment tweet-id 1) (car/setbit location tweet-id 1) (car/setbit created-at tweet-id 1))))
find value for keyfind bit of index
set bit to 1
(wcar* (car/bitcount "SCOTLAND") (car/bitcount "JOVIALITY") (car/bitcount "26062014114532"))
ENG
SCT
WAL
NIR
valuekey
ENG 1
SCT 0
WAL 0
NIR 0
(car/setbit “ENG” 0 1)
tweet-idkey
valuekey
ENG 0 1
SCT 1 0
WAL 0 0
NIR 0 0
(car/setbit “SCT” 1 1)
tweet-idkey
valuekey
ENG 1 0 1
SCT 0 1 0
WAL 0 0 0
NIR 0 0 0
(car/setbit “ENG” 2 1)
tweet-idkey
valuekey
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
(car/setbit “SCT” 15 1)
tweet-idkey
valuekey
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
(car/bitcount "ENG")
8
valuekey
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
(car/bitcount "SCT")
84
valuekey
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
(car/bitcount "WAL")
843
valuekey
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
(car/bitcount "NIR")
8431
valuekey
JOVIAL
SHY
HOSTILE
FATIGUE
valuekey
JOVIAL 1
SHY 0
HOSTILE 0
FATIGUE 0
(car/setbit “JOVIAL” 0 1)
tweet-idkey
valuekey
JOVIAL 0 1
SHY 0 0
HOSTILE 1 0
FATIGUE 0 0
(car/setbit “HOSTILE” 1 1)
tweet-idkey
valuekey
JOVIAL 0 0 1
SHY 1 0 0
HOSTILE 0 1 0
FATIGUE 0 0 0
(car/setbit “SHY” 2 1)
tweet-idkey
valuekey
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
SHY 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
(car/setbit “JOVIAL” 15 1)
tweet-idkey
valuekey
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
SHY 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
(car/bitcount [*])
8134
valuekey
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))
How many people in England are Happy?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))
How many people in England are Happy?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))
How many people in England are Happy?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))
How many people in England are Happy?
How many people in England are Happy?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
How many people in England are Happy?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
How many people in England are Happy?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
1
AND
How many people in England are Happy?
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 1
AND
How many people in England are Happy?
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 0 1
AND
How many people in England are Happy?
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 1
AND
How many people in England are Happy?
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 1
How many people in England are Happy?
(car/bitcount “ENGLAND&JOVIALITY")
4
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))
How many people in Scotland are Tired and Grumpy?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))
How many people in Scotland are Tired and Grumpy?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))
How many people in Scotland are Tired and Grumpy?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))
How many people in Scotland are Tired and Grumpy?
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH
OR
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0
OR
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 1 0
OR
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
OR
How many people in Scotland are Tired and Grumpy?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
OR
How many people in Scotland are Tired and Grumpy?
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
How many people in Scotland are Tired and Grumpy?
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
AND
How many people in Scotland are Tired and Grumpy?
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 00 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
How many people in Scotland are Tired and Grumpy?
2
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
adamwynne/twitter-api
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
clojurewerkz/meltdown
LMAX-Exchange/disruptor
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
dakrone/clojure-opennlp
“I’m loving #EuroClojure! ”
Positive Affect: enthusiastic, active, alert !
Negative Affect: subjective distress !
Emerged as distinctive, orthogonal dimensions
PANAS
1
23
4
51
3153
2335345221
PANAS
1
23
4
51
3153
2335345221
1 3 5 9 10 12 14 16 17 19
2 4 6 7 8 11 13 15 18 20
PANAS
1
2345
1
31
53
233534
5 22 1
1 3 5 9 10 12 14 16 17 19
2 4 6 7 8 11 13 15 18 20
39 19
positive affect! negative affect
PANAS-xGeneral Dimension Scales!Negative Affect (10) Positive Affect (10)
Basic Positive Emotions Scales!Joviality (8) Self-assurance (6) Attentiveness (4)
Basic Negative Emotions Scales!Fear (6) Hostility (6) Guilt (6) Sadness (5)
Other Affective States!Shyness (4) Fatigue (4) Serenity (3) Surprise (3)
PANAS-t
Accounts for bias on social media!
Outlines sanitisation
Validate against 10 real events
Sanitisation
• Exclude media/advertising/spam
• Account for text speak
• Account for smileys & emoj’s
• Word stemming (or lemmatisation)
• Part of Speech Tagging (POS)
LMAO
GetRichQuick.com
dakrone/clojure-opennlp
SHYNESS FATIGUE
SERENITY SURPRISE
FEAR HOSTILITY
GUILT SADNESS
JOVIALITY SELF ASSURANCE ATTENTIVENESS
We have sentiment key
s!
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
mbostock/d3
[-75.14310264, 40.05701649]
Reverse Geocoding Issues• Don’t want external services
• Don’t want heavy IO
• Don’t want round trips to the database
• Accuracy not too much of a concern
Equirectangular Projection
Courtesy of Eric Gaba (Wikimedia Commons User: Sting)
x = λcosφ1 y = φ
Tissot's indicatrix of deformation
Courtesy of Eric Gaba (Wikimedia Commons User: Sting)
x (px)
y (px)
x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!
x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!
x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!
x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!
x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!
x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!
Server side! (JavaFX)
Mike Bostock & Nick Rabinowitz
(:require [hiccup.page :as hiccup]) (:import (javafx.scene.web WebView)) !(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"]) !(defn create-browser [html] (-> WebView. .getEngine (.loadContent html))) !(defn build-page [script-paths] (hiccup/html5 [:head (for [path script-paths] [:script (slurp (io/resource path))])])) !(defn coordinates->name [longitude latitude] (let [browser (-> scripts build-page create-browser)] (.executeScript browser (str "geo.decode([" longitude "," latitude "])"))))
JavaFX
(:require [hiccup.page :as hiccup]) (:import (javafx.scene.web WebView)) !(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"]) !(defn create-browser [html] (-> WebView. .getEngine (.loadContent html))) !(defn build-page [script-paths] (hiccup/html5 [:head (for [path script-paths] [:script (slurp (io/resource path))])])) !(defn coordinates->name [longitude latitude] (let [browser (-> scripts build-page create-browser)] (.executeScript browser (str "geo.decode([" longitude "," latitude "])"))))
JavaFX
(:require [hiccup.page :as hiccup]) (:import (javafx.scene.web WebView)) !(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"]) !(defn create-browser [html] (-> WebView. .getEngine (.loadContent html))) !(defn build-page [script-paths] (hiccup/html5 [:head (for [path script-paths] [:script (slurp (io/resource path))])])) !(defn coordinates->name [longitude latitude] (let [browser (-> scripts build-page create-browser)] (.executeScript browser (str "geo.decode([" longitude "," latitude "])"))))
JavaFX
(:require [hiccup.page :as hiccup]) (:import (javafx.scene.web WebView)) !(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"]) !(defn create-browser [html] (-> WebView. .getEngine (.loadContent html))) !(defn build-page [script-paths] (hiccup/html5 [:head (for [path script-paths] [:script (slurp (io/resource path))])])) !(defn coordinates->name [longitude latitude] (let [browser (-> scripts build-page create-browser)] (.executeScript browser (str "geo.decode([" longitude "," latitude "])"))))
JavaFX
(:require [hiccup.page :as hiccup]) (:import (javafx.scene.web WebView)) !(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"]) !(defn create-browser [html] (-> WebView. .getEngine (.loadContent html))) !(defn build-page [script-paths] (hiccup/html5 [:head (for [path script-paths] [:script (slurp (io/resource path))])])) !(defn coordinates->name [longitude latitude] (let [browser (-> scripts build-page create-browser)] (.executeScript browser (str "geo.decode([" longitude "," latitude "])"))))
Stuart Sierras
Component lib!!
JavaFX
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
KirinDave/clj-time
“Thu Jun 26 14:35:57 +0000 2014”
over 86 thousand seconds in a dayover 31 million seconds in a year
mm:260620141435
HH:260620141400
Monday Tuesday Wednesday Thursday Friday Saturday Sunday
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 27 28 29
30 31
dd:260620140000
JAN FEB MAR APR
MAY JUL AUG
SEP OCT NOV DEC
MM:010620140000
2013
2015
yy:010120140000
(:require [clj-time.format :as time]) !
!
(def input-formatter "EE MMM d HH:mm:ss Z yyyy") (def formats { "mm:" "ddMMyyHHmm" "HH:" "ddMMyyHH00" ... }) !
!
(defn create-key [date [prefix formatter]] (let [out-date (time/unparse formatter date)] (str prefix out-date))) !
!
(defn date->keys [created_at] (let [date (time/parse input-formatter created_at)] (map #(create-key date %) formats)))
(:require [clj-time.format :as time]) !
!
(def input-formatter "EE MMM d HH:mm:ss Z yyyy") (def formats { "mm:" "ddMMyyHHmm" "HH:" "ddMMyyHH00" ... }) !
!
(defn create-key [date [prefix formatter]] (let [out-date (time/unparse formatter date)] (str prefix out-date))) !
!
(defn date->keys [created_at] (let [date (time/parse input-formatter created_at)] (map #(create-key date %) formats)))
(:require [clj-time.format :as time]) !
!
(def input-formatter "EE MMM d HH:mm:ss Z yyyy") (def formats { "mm:" "ddMMyyHHmm" "HH:" "ddMMyyHH00" ... }) !
!
(defn create-key [date [prefix formatter]] (let [out-date (time/unparse formatter date)] (str prefix out-date))) !
!
(defn date->keys [created_at] (let [date (time/parse input-formatter created_at)] (map #(create-key date %) formats)))
RINGSyncing
Journaling
Business logic
What
Where
When
Mem
ory
Web
API
Incoming Data
ptaoussanis/carmine
SENTIMENT !
LOCATION !
mm:date-time HH:date-time dd:date-time MM:date-time yy:date-time
We have 11 keys in total!
What
Where
When
(:require [taoensso.carmine :as redis]) !
!
(defn create-tweet-id [] (redis/incr "global:tweet.id")) !
!
(defn set-bits [data-keys tweet-id] (doseq [data-key data-keys] (redis/setbit data-key tweet-id 1))) !
!
(defn data->analytics [data-keys] (let [tweet-id (create-tweet-id)] (set-bits data-keys tweet-id)))
(:require [taoensso.carmine :as redis]) !
!
(defn create-tweet-id [] (redis/incr "global:tweet.id")) !
!
(defn set-bits [data-keys tweet-id] (doseq [data-key data-keys] (redis/setbit data-key tweet-id 1))) !
!
(defn data->analytics [data-keys] (let [tweet-id (create-tweet-id)] (set-bits data-keys tweet-id)))
(:require [taoensso.carmine :as redis]) !
!
(defn create-tweet-id [] (redis/incr "global:tweet.id")) !
!
(defn set-bits [data-keys tweet-id] (doseq [data-key data-keys] (redis/setbit data-key tweet-id 1))) !
!
(defn data->analytics [data-keys] (let [tweet-id (create-tweet-id)] (set-bits data-keys tweet-id)))
{ 'query-type': 'count', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
weavejester/compojure
JSON query
{ 'query-type': 'count', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
weavejester/compojure
JSON query
/api?_query={json}
(defmulti execute :query-type) !!(defmethod execute “count" [] … logic here) !!(defmethod execute “p-affect“ [] … logic here) !!(defmethod execute “n-affect“ [] … logic here)
{ 'query-type': 'count', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
weavejester/compojure
JSON query
/api?_query={json}
(defmulti execute :query-type) !!(defmethod execute “count" [] … logic here) !!(defmethod execute “p-affect“ [] … logic here) !!(defmethod execute “n-affect“ [] … logic here)
{ 'query-type': 'count', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
weavejester/compojure
JSON query
/api?_query={json}
(defmulti execute :query-type) !!(defmethod execute “count" [] … logic here) !!(defmethod execute “p-affect“ [] … logic here) !!(defmethod execute “n-affect“ [] … logic here)
weavejester/compojure
JSON query
/api?_query={json}
{ 'query-type': 'count', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
(defmulti execute :query-type) !!(defmethod execute “count" [] … logic here) !!(defmethod execute “p-affect“ [] … logic here) !!(defmethod execute “n-affect“ [] … logic here)
weavejester/compojure
JSON query
/api?_query={json}
{ 'query-type': ‘p-affect', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
(defmulti execute :query-type) !!(defmethod execute “count" [] … logic here) !!(defmethod execute “p-affect“ [] … logic here) !!(defmethod execute “n-affect“ [] … logic here)
weavejester/compojure
JSON query
/api?_query={json}
{ 'query-type': ‘p-affect', 'params': { 'sentiments': [ 'FATIGUE', 'HOSTILITY' ], 'locations': [ 'SCT' ] } }
The code for this project is about to be open sourced
!
We would love to have more contributors! !
To find out more, follow: #APassionForCode