workshop: data visualization for corpus linguistics via shiny framework

Data Visualizationfor CorpusLinguistics:

Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Data Visualization for Corpus Linguistics:Shiny Framework

Olga Scrivner


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

What You Will Learn

1. Introduction to Visual Analytics

2. Reactive Web Framework

3. Shiny Application

4. Practice

https://languagevariationsuite.wordpress.com/



Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Data Analytics

“Analytics is the critical technologyneeded to bring value out of data”

(Anonymous)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visual Analytics

“The science of analytical reasoningfacilitated by visual interactive interfaces”

(Thomas and Cook, 2005)

“Visual analytics integrates new computational andtheory-based tools with innovative interactive techniquesand visual representations to enable human-information

discourse” (Thomas and Cook, 2005)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visual Analytics and Linguistics

“ Language is a system of relatively arbitrary symbols”(Baker-Shenk and Cokely, 1980)

http://www.tableaufit.com/tableau-conference-linguistics-data-visualization/

http://www.tableaufit.com/tableau-conference-linguistics-data-visualization/


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visual Analytics (and Corpus Linguistics)

1. Exploration (Queries)

2. Comprehension (Analysis)

3. Communication (Presentation)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Corpus Linguistics Visualization

http://scimaps.org/home.html

http://scimaps.org/home.html


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Visualization Tools

Challenges:

I Many tools are built for non-linguistic research

I Limited preprocessing

I Visualization may not fit linguistic data

Solution: Reactive Framework

I Interactive and web-based

I Research-driven customization

I Build your own tool!


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Web Framework

“Reactive Systems are highly responsive, giving userseffective interactive feedback”

http://www.reactivemanifesto.org/

http://littleactuary.github.io/blog/Web-application-framework-with-Shiny/

http://www.reactivemanifesto.org/



Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Architecture


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Framework and Data Science

“The impact of data scientists’ work depends on how wellothers can understand their insights to take further actions”

Benefit 1: Interactive display and manipulation of data

Benefit 2: No installation required

Benefit 3: Easy to develop and share with clients andproject teams

Benefit 4: Open source library

http://datascience.ibm.com/blog/shiny-a-data-scientist-best-friend/

http://datascience.ibm.com/blog/shiny-a-data-scientist-best-friend/


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Application

1. Shiny is an R package for building interactive webapplications

2. Open-Sourced by RStudio 11/2012 on CRAN

3. Uses web sockets (new HTTP):

I Interactive communication sessions between the user’sbrowser and a server without having to poll the serverfor a reply

4. Entirely extensible - custom input/output


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Library




Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Gallery - Get Inspired

https://www.rstudio.com/products/shiny/shiny-user-showcase/

https://www.rstudio.com/products/shiny/shiny-user-showcase/


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Interactive Text Mining Suite (Scrivner et al.2016)

1. Web application for text processing and mining

2. Interactive natural language processing techniques

I Wordstops, stemming, text-preprocessing

3. High customization


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Stopwords

Stopwords (e.g. the, and):


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Manual Removal of Stopwords


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Stemming


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Word Cloud Representation


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Customization


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Format Flexibility

Example: Using Google Books API

Current limitation is 40 books


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Workshop Materials

1. Rstudio

2. R

3. Shiny library

4. Materials:

https:

//languagevariationsuite.wordpress.com/




Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

R software

R is a free software for data analysis, text mining andvisualization.

To install R on Window:

1. Download the binary file for Rhttps://cran.r-project.org/bin/windows/base/

R-3.3.1-win.exe

2. Open the downloaded .exe file and Install R

To install R on Mac:

1. Download the appropriate version of .pkg filehttps://cran.r-project.org/bin/macosx/

2. Open the downloaded .pkg file and Install R

https://cran.r-project.org/bin/windows/base/R-3.3.1-win.exe

https://cran.r-project.org/bin/windows/base/R-3.3.1-win.exe

https://cran.r-project.org/bin/macosx/


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

R Studio

RStudio is a free user interface for R.

1. Install the appropriate RStudio version https:

//www.rstudio.com/products/rstudio/download/

2. Run it to install R-studio

https://www.rstudio.com/products/rstudio/download/

https://www.rstudio.com/products/rstudio/download/


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

R Studio Structure


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Installing Packages

In your bottom left window - go to Packages


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Selecting Packages - shiny


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Creating Scripts

Create R File: File → New File → R Script


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Execution - RUN

To execute - click run

In the script (top left window) type:

library(shiny)

Click Run (your cursor can be at the beginning or at the endof the line)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Demo

runExample(“01 hello”)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Shiny Demo


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

UI.R


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Input/Output

I Input - things user can toggle

I Output - R objects that user can see, often depend oninputs


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Closing App


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Step 1 - Create New Web Shiny App


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Step 2 - Run App


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

UI

shinyUI( fluidPage(

titlePanel(“Old Faithful Geyser Data”),

sidebarLayout(

sidebarPanel(

sliderInput(“bins”,

“Number of bins:”,

min = 1,

max = 50,

value = 30)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Modifying UI - Practice

I Change slider’s label:Number of bins → Choose a number

I Change slider’s values: max and value

I Save

I RunApp


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

HTML

The UI script simply creates HTML


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

HTML

HTML tags:

http:

//shiny.rstudio.com/articles/tag-glossary.html

I h1() = header1

I br() = line break

I p() = paragraph

I hr() = line

http://shiny.rstudio.com/articles/tag-glossary.html

http://shiny.rstudio.com/articles/tag-glossary.html


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Adding HTML Tags to UI.R - Practice

shinyUI( fluidPage(

titlePanel(“My Title”),

h3(“My subtitle”),

p(“This is my first app!”),

br(),

hr(),


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Layout

http:

//shiny.rstudio.com/articles/layout-guide.html

http://shiny.rstudio.com/articles/layout-guide.html

http://shiny.rstudio.com/articles/layout-guide.html


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

mainPanel

mainPanel( plotOutput(”distPlot”))

Let’s add 3 tab panels: Plot, Summary, Table


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

tabsetPanel

mainPanel(

tabsetPanel(

tabPanel(“Plot”, plotOutput(“distPlot”)),

tabPanel(“Summary”),

tabPanel(“Table”)

)

)


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Input Data


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Input Data

1. Blog https:

//languagevariationsuite.wordpress.com/

2. Download csv file - movie metadata.csv

3. Place this file into the directory myshiny




Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Input in UI.R

I Let’s look at the fileInput() function from ShinyReference page

I Type in your browser: fileInput Shiny

I or go to the Shiny Reference page -https://shiny.rstudio.com/reference/shiny/

latest/fileInput.html

https://shiny.rstudio.com/reference/shiny/latest/fileInput.html

https://shiny.rstudio.com/reference/shiny/latest/fileInput.html


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Input in UI.R

I Scroll down to Examples

I We will add fileInput function inside sidebarPanel

I Copy fileInput function


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Input in UI.R

I Paste it after slider

NB: commas are important!


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Reactive Function in Sever.R

Reactive - it changes every time the user uploads new data

I Create a function that reads csv file

I myfile() is a function that will read and return csv files


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Render - Reactive Data


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Function summary in Server.R

output$summary <- renderPrint({

summary(myfile())

})


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

New Output Function table in Server.R

output$table <- renderDataTable({

myfile()

})


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Linking summary and table with UI.R

tabsetPanel(

tabPanel("Plot", plotOutput("distPlot")),

tabPanel("Summary", verbatimTextOutput("summary")),

tabPanel("Table", dataTableOutput("table"))

)

RunApp


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Testing CSV Upload


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

One more IF-Statement - Server.R

We want to do a histogram for csv file:

if (condition) {do..} else {do...}

output$distPlot <- renderPlot({

if (is.null(input$file1)) {

....

hist(x, breaks = bins, col = ’darkgray’, border =

’white’)

}

else{

x <- myfile()$budget

bins <- seq(min(x), max(x), length.out = input$bins

+ 1)

hist(x, breaks = bins, col=’red’,

main = ’My New Histogram’)

}

})


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

One more IF-Statement - Server.R

We want to do a histogram for csv file:

if (condition) {do..} else {do...}output$distPlot <- renderPlot({

if (is.null(input$file1)) {

....

hist(x, breaks = bins, col = ’darkgray’, border =

’white’)

}

else{

x <- myfile()$budget

bins <- seq(min(x), max(x), length.out = input$bins

+ 1)

hist(x, breaks = bins, col=’red’,

main = ’My New Histogram’)

}

})


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

RunApp


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Server - Recap


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Deployment Options

1. Share server.r and ui.r

2. Host on shinyapps.io

3. Host on Shiny server


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Deploy with shinyapps.io

I www.shinyapps.io

I sign up for an account.

I Publish Application button in RStudio and followinstructions

www.shinyapps.io


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Deploy with shinyapps.io


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

My shinyapps.io


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Future of Visualization Tools: CNS

Cyberinfrastructure for Network Science Center at IndianaUniversity - http://cns.iu.edu/

Our current project on Shiny Framework (team OlgaScrivner and Jivitesh Poojary)

I Build Shiny templates for Data Visualization: stage I

https:

//github.com/Jivitesh-Poojary/CNS-Shiny-Apps

I Create user-friendly customizable Shiny dashboards:stage II

I Develop preprocessing plugins for Shiny templates:stage III

http://cns.iu.edu/

https://github.com/Jivitesh-Poojary/CNS-Shiny-Apps

https://github.com/Jivitesh-Poojary/CNS-Shiny-Apps


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Useful resources

1. Shiny official tutorial -http://shiny.rstudio.com/tutorial

2. Cheat sheet - http://shiny.rstudio.com/images/shiny-cheatsheet.pdf

3. Publish your app free - http://www.shinyapps.io

4. Examples -http://www.showmeshiny.com/

5. Tutorial by Dean Attali - http://deanattali.com/blog/building-shiny-apps-tutorial/

http://shiny.rstudio.com/tutorial

http://shiny.rstudio.com/images/shiny-cheatsheet.pdf

http://shiny.rstudio.com/images/shiny-cheatsheet.pdf

http://www.shinyapps.io

http://www.showmeshiny.com/

http://deanattali.com/blog/building-shiny-apps-tutorial/



Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Thank you!

My email: [email protected]


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

References

Borner, Katy. 2015. Atlas of Knowledge: Anyone Can Map. TheMIT PressCollins, Christopher. 2012. Bridging the Linguistic VisualizationDivide. LingVis/UNCLH WorkshopGrolemund, Garrett. 2017. How to start with Shiny, Part 1.Workshop


Shiny Framework

Olga Scrivner

Visual Analytics

ReactiveFramework

Shiny App

Practice

Credits

http:

//deanattali.com/blog/building-shiny-apps-tutorial/

http://scimaps.org/mapdetail/stream_of_scientific_128

https://github.com/IBMDataScience/dsx-shiny-apps

http://www.slideshare.net/SarahAerni/

data-science-as-a-commodity-use-madlib-r-other-oss-tools-for-data-science-from-pivotal-open-source-hub-meetup

http://www.unixstickers.com/image/data/stickers/

react/badge/React-JS.sh.png

https://github.com/rstudio/shiny/issues/250

http://www.slideshare.net/ilio-catallo/

spring-mvc-the-basics



http://scimaps.org/mapdetail/stream_of_scientific_128

https://github.com/IBMDataScience/dsx-shiny-apps

http://www.slideshare.net/SarahAerni/data-science-as-a-commodity-use-madlib-r-other-oss-tools-for-data-science-from-pivotal-open-source-hub-meetup

http://www.slideshare.net/SarahAerni/data-science-as-a-commodity-use-madlib-r-other-oss-tools-for-data-science-from-pivotal-open-source-hub-meetup

http://www.unixstickers.com/image/data/stickers/react/badge/React-JS.sh.png

http://www.unixstickers.com/image/data/stickers/react/badge/React-JS.sh.png

https://github.com/rstudio/shiny/issues/250

http://www.slideshare.net/ilio-catallo/spring-mvc-the-basics

http://www.slideshare.net/ilio-catallo/spring-mvc-the-basics

workshop: data visualization for corpus linguistics via shiny framework

Data & Analytics