big data: data analysis boot camp starting hadoop on linuxccartled/teaching/2018... · 1/19 intro....

19
1/19 Intro. Accessing Linux One time command Everytime commands Sometime commands Big Data: Data Analysis Boot Camp Starting Hadoop on Linux Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD Chuck Cartledge, PhD 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018 19 January 2018

Upload: others

Post on 22-May-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

1/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Big Data: Data Analysis Boot CampStarting Hadoop on Linux

Chuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhDChuck Cartledge, PhD

19 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 201819 January 2018

Page 2: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

2/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Table of contents (1 of 1)

1 Intro.

2 Accessing LinuxStep-by-step

3 One time command

4 Everytime commands

Start and stop.5 Sometime commands

Page 3: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

3/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

What are we going to cover?

Hadoop provides the map-reduceinfrastructure used by selected R scriptsin this boot camp. We will showcommands that:

1 Are executed once,

2 Must be executed everytime, and

3 Should be executed everytime.

Page 4: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

4/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Finding the Linux VM

Locate the hadoop shortcut onthe desktop

Find and start the LinuxVMware.

Page 5: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

5/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Same image.

Find and start the Linux VMware.

Page 6: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

6/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Starting the Linux VM

1 Locate the Hadoop label onthe left bar

2 Press the run button on thetop

3 Accept the various defaultwarnings and settings

Find and start the Linux VM.

Page 7: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

7/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Same image.

Find and start the Linux VM.

Page 8: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

8/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Log into the Linux/Ubuntu software

Login name is:ODUBootcamp

Password is: ODUBootcam

You will need the password manytimes.

Page 9: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

9/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Same image.

You will need the password many times.

Page 10: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

10/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Bring up a command/terminal window

Double click the terminal icon.

You have now accessed the LinuxVM and brought up a terminal

window.

Page 11: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

11/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Step-by-step

Same image.

You have now accessed the Linux VM and brought up a terminalwindow.

Page 12: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

12/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Format the Hadoop namenodes

In a terminal window; type thiscommand:/usr/local/hadoop/bin/hdfs

namenode -format

and press the Enter key.

This will generate lots of outputyou can ignore.

Page 13: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

13/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Same image.

This will generate lots of output you can ignore.

Page 14: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

14/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Start and stop.

Start the Hadoop applications

In a terminal window; type thiscommand:/usr/local/hadoop/sbin/start-all.sh

and press the Enter key.

You will be asked for theODUBootcamp password 4

times.

Page 15: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

15/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Start and stop.

Same image.

You will be asked for the ODUBootcamp password 4 times.

Page 16: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

16/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Start and stop.

Stop the Linux/Ubuntu software.

Locate and press the gear buttonin the upper left corner.

Press the “ShutDown. . . ”option, then the “Shut Down”

button.

Page 17: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

17/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Start and stop.

Same image.

Press the “ShutDown. . . ” option, then the “Shut Down” button.

Page 18: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

18/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Stop the Hadoop applications

In a terminal window; type thiscommand:/usr/local/hadoop/sbin/stop-all.sh

and press the Enter key.You will be asked for theODUBootcamp password 4times.

You should stop the Hadoopapplications in a controlled

manner.

Page 19: Big Data: Data Analysis Boot Camp Starting Hadoop on Linuxccartled/Teaching/2018... · 1/19 Intro. Accessing LinuxOne time command Everytime commandsSometime commands Big Data: Data

19/19

Intro. Accessing Linux One time command Everytime commands Sometime commands

Same image.

You should stop the Hadoop applications in a controlled manner.