Upload File

w w , ] p z w ( } u v } u µ ] v p ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · title: microsoft...

  • Home
  • Documents
  • W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM
14
BioHPC: A High Performance Computing System for Life Sciences Bioinformatics Facility Biotechnology Resource Center Cornell University

Upload: others

Post on 02-Aug-2020

1 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

BioHPC: A High Performance Computing System for Life Sciences

Bioinformatics FacilityBiotechnology Resource Center

Cornell University

Page 2: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

1. Hosted Servers: Computers owned by individual labs but managed by the BioHPC team;2. Rental Servers: Computers available for hourly rent by BioHPC users;3. BioHPC Network Storage: Central storage accessible from each BioHPC server;4. Login Nodes: Computers used for accessing BioHPC data storage without reservation, and for BSCB users to

access their cluster. The login nodes can also be used for accessing compute servers from outside campus without VPN. No computing is allowed on these machines.

Hosted Servers 1 Rental Servers 2 BioHPC Storage 3Login Nodes 4

Network

The BioHPC Cloud has 4 types of computers

cbsulogin

cbsulogin2

cbsulogin3

cbsuvitisgen2

cbsubscb01

cbsuecco1

cbsuxxxx1

cbsumm01

cbsulm01

cbsum1c1b001

cbsugpu02…… ……

Page 3: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

On each server,• Your home directory /home/<userID> is located on the BioHPC network

storage. You can access the same home directory from any BioHPC computer. However, it is slower to read/write files in the home directory than the local storage. Do not compute directly on files in home directory.

• Local storage: There is a /workdir on each server which is mounted on the local hard drives inside each server. It is much faster and much more reliable to access data files in the /workdir. Make sure to copy data files from your home directory to the /workdir before computing.

Hard drive/home/az999

/workdir

cbsumm01

Page 4: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

• Your home directory: Up to 200 GB free storage for each account. You can purchase more storage if needed. The data files in your home directory are kept as long as your account is active.

• The working directory /workdir: There is no restriction on size, up to the local storage capacity of each server. On rental machines, the /workdiris cleared after your reservation ends. On hosted machines, files on /workdir are kept permanently.

Limit on storage size

Page 5: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Data backup:

• By default, there is no second copy or backup for files in your home directory. As the BioHPC storage system uses RAID, the risk of data loss due to hard drive failure is extremely low. However, if files are deleted due to human errors, or your account gets compromised, they cannot be recovered.

• For important data files, we recommend that you keep a second copy. BioHPC does provide optional data backup service on user-specified directories. More information on the backup service can be found at https://biohpc.cornell.edu/lab/userguide.aspx?a=backupguide .

Page 6: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

The Firewall

• The three login nodes (cbsulogin, cbsulogin2, cbsulogin3) are open to the external network. You can access these three machines from anywhere in the world.

• The rest of BioHPC servers are behind the Cornell firewall. On the Cornell campus, they can be accessed directly. From outside Cornell, you can either connect through Cornell VPN (https://it.cornell.edu/cuvpn ), or “ssh” via the login nodes.

Page 7: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Use the BioHPC Website to Manage User Accesshttps://biohpc.cornell.edu

Reserve a Rental Machine Change Password Make Payment

Mange VNC Storage & Backup Book Office Hours

Page 8: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Quick start Step 1. Transfer data files to BioHPCYou can use one of the login nodes or any other BioHPC machines to transfer data files to your home directory. Any sftp client software will work, e.g. FileZilla (https://filezilla-project.org/download.php?show_all=1 )

Page 9: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Quick start Step 2. Reserve a rental machine (skip this step if you are using a hosted machine)

1. Login to the BioHPC website (https://biohpc.cornell.edu ) and click “Reservations”;2. Reserve a computer from one of the categories. Each category is priced differently

(https://biohpc.cornell.edu/lab/Pricing.aspx ) ;3. You can only access a machine while your reservation is active. 4. Reserve time longer than what is needed to run your job. You can adjust or cancel a reservation at

any time through the “My Reservations” page.

Page 10: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Quick start Step 3. Login to the computer and run software

1. Login to the compute server through ssh (e.g., Putty on Windows, Terminal on Mac);2. Create your working directory under /workdir and copy your data files there;3. You can find all pre-installed software on this page:

https://biohpc.cornell.edu/lab/labsoftware.aspx (Use “*” as wildcard when searching for a software name). Follow instructions on each software detail page;

4. If you do not find the software you want to use, email [email protected] or install the software by yourself in your home directory;

5. If you are using a rental machine, make sure to copy results back to home directory after the work is done. You can always access your home directory through the login nodes.

Page 11: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Miscellaneous information

• When using software like Python, R, Java, GCC, Docker, Tensorflow, etc. for the first time, please consult software details page under https://biohpc.cornell.edu/lab/labsoftware.aspx (use the search box to find the entry)

• Please read our user guide to find more information about data file sharing, using GUI software, etc.: https://biohpc.cornell.edu/lab/userguide.aspx

Page 12: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Miscellaneous information

• Use “screen” when running software;

• Parallelization – running jobs on multiple cores:

perl_fork_univ.pl: perl_fork_univ.pl jobList 5 GNU parallel Local scheduler (SLURM)

Page 13: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

Where to find more information?• User guide https://biohpc.cornell.edu/lab/userguide.aspx• Workshops https://biohpc.cornell.edu/workshops.aspx

Prefer to talk to a real person? Book an office hour https://biohpc.cornell.edu/lab/office1.aspx

More questions?Email: [email protected]

How to get help ?

Page 14: W W , ] P Z W ( } u v } u µ ] v P ^ Ç u ( } > ] ( ^ ] v · 2019-03-15 · Title: Microsoft PowerPoint - onboarding.pptx Author: jp86 Created Date: 3/15/2019 5:57:41 PM

The BioHPC [email protected]


dK s/Zdh > Z /E E W W ] v ] o r Z } u u v ] } v

v W } v } u ] d ] u D s l D W E ] µ o ] } v W E - SRF

v Z ] ( ' U ^dW X U D X^ X ~'^ V ~KZ / V ~^ ] v W } P u ... · W v P µ Z W u ] v h v } o v ^ µ Z µ W v Ç ] u v v d Z D µ µ v D ^ ] u v µ Z : µ l ^ ] u ~ ] µ v } ] o ] > }

u ] W } u } ] } v o ] ] } v & } u t E } v W } ( } ] o d Z

v P ] v ] v P r ] Ç o v W ] v ] o / u } À u v W o v

v Z v ] v P ] À ] Ç ] v Z ] ] W } ( ] } v W with Disabilities Par… · v v µ o ] ] o ] Ç ^ ] ] ^ µ o u v V µ Z u U E, W h v ] À ] Ç } ( E Á , u Z ] U / v ] µ } v ] ] o

K ( ( ] } u u µ v ] ] } v W

&KK & } > v } µ v Ç } u u µ v ] Ç v } v } v P P u v ... · Z ^WKE^/ > dK W ] } } ( À o } u v > ^^ W Æ u ^ > Zz W & µ o o ] u U v } v r Æ u ¨ í õ X ì ì l Z } µ X , o

^ µ } ] v P ] v ( } u ] } v · 2019. 7. 31. · ^ í ^ µ } ] v P ] v ( } u ] } v ^ Ç v Z ] } ( Ç u u ] ^ W } o Ç u Ç W } r } o Ç u ] ] } v D } ] ( ] ] } v v W Z } } u ] W }

W W W o v Ç o l ~ ñ ì u ] v U / W / v À ] o l ~ î ñ u ] v U K W K o v ] } v …ecl.snu.ac.kr/NFUpload/nfupload_down.php?tmp_name... · 2017-01-05 · Z ] Ç } ( Z u } v ] Æ

u } ] } v } } À ] v } ] } v ( } · 2020. 10. 1. · D ] o / v } u W Z v } u W v Z W } ~W Z v } u W v Z í ó : µ o Ç î ì í ñ D Z W l l Á Á Á X Z v } u v Z } X } u l µ ]

W } u ] ^ Z o } ( / v ( } u ] } v ~ZK

ÀK W ] u W ^ µ ] v W ] ( } u } ' } À v u v Primer_1.pdf · ÀK W ] u W ^ µ ] v W ] ( } u } ' } À v u v ... l

^ u o o µ ] o ] v P W ] ] ] } v > } v W } P u...h August 2020 ^ u o o µ ] o ] v P W ] ] ] } v > } v W } P u, Z [ ^ u o o µ ] o ] v P W ] ] ] } v > } v W } P u } À ] P } i ( ] v

d Z - GAIL · E u } ( Z K P v ] Ì ] } v W ' /> ~/ v ] > ] u ] t ] > ] v l W Á Á Á X P ] o } v o ] v X } u D ] v ] Ç l u v W D ] v ] Ç } ( W } o µ u v E µ o '

0LQLPXP 5HTXLUHPHQWV 6HQWLQHO 6DIH :RUN 3ODQQLQJ … · / v µ Ç D ] v ] u µ u Z µ ] u v U ^ v ] v o U ^ ( t } l W o v v ] v P v W o v K ] } v ^ Z u u } µ o

A a A a A a - Charisse Yu · 2020. 8. 3. · 6 V U u U u U u Charisseyu.com udon noodles. 7 W V v V v V v Charisseyu.com vienna sausage. 8 X W w W w W w Charisseyu.com watermelon

W } ] o ] Ç · Ç Z µ o W u v t^ } v ~W ð u } W W u v t^ } v tE } o ] ( ~> í ð í

u u ] o v v ] W } µ ] } v ] v ] ] Z } o µ u ] W À ] o o ......r X X & X

W o v v d } o / u o u v ] v P À o } ^ } W } i / ] ] } v Æ

W o v } W ^ ] v P Á } l } } v ( ] r } u U ( ] r À ] X Forms/2019-Rex... · W o v } W ^ ] v P Á } l } } v ( ] r } u U ( ] r À ] X ... x

Æ u o & o } ] v P W } ] v & } u

~ ' } À v u v } ( D Z Z v ] } ( , v ] Z Z } ( u Ç ^ } W ] u ' } À …...~ ' } À v u v } ( D Z Z v ] } ( , v ] Z Z } ( u Ç ^ } W ] u ' } À X , ] P ... ... K v

W µ o ] ^ À ] } u u ] ] } v U t v P opscwbapplication.in/pdf20/QUALIFIED_ROLL_NO_WISE_LIST.pdf · W µ o ] ^ À ] } u u ] ] } v U t v P o. W µ o ] ^ À ] } u u ] ] } v U t v P

^ µ u u Ç W ^ W o v ( } Z / u o u v ] } v } ( Z D µ r À ] W

& Z,' v o , } ] o v W } v o h À ] W v o À ] } Ç } u u ] D

W µ o ] ] v l ] v P t W o o } u u µ v - Texas …...W µ o ] ] v l ] v P t W o o } u u µ v ] Ç t ^ Ç u ~ t^ ] v , µ ] v , À Ç / u ... W µ o ] ] v l ] v P t W o o } u u µ

IR presentation updated 03apr2020...ñ u } v P Z u } ( ( ] ] v d^K Á ] Z µ ] } À ] µ o ] Ç ^ E W ^ v ] v À ] V hZ W µ } V ^W W ] v ^ } µ Z W ] ( ] V E^ W E } Z v ^ } µ Z u

D v } Ç W DW h · ^d d W ] ] v } v ] ] } v U ] ( o ] o D v } Ç W DW h W DW h o u > ] v } ( Z } } ( D ] o Æ u ] v u µ Z l Z W DW Á ] Ç ] (

] Ç } Ì v W d Á o À } u u } v D ] / v ] } v W ] ( o o ] v ... KDDMetricInterpretationPitfalls.pdf] Ç } Ì v W d Á o À } u u } v D ] / v ] } v W ] ( o o ] v î á

D ] v ] Ç } ( ^ ] ] W } P u u / u o u v ] } v E ] } v o ^ ] ] o K ( ( ] ï · 2020. 9. 25. · DK^W/ W W r ì õ l î ì î ì ~& W P ï } ( ó î X î u o } Ç [ ^ / v µ v ^ Z

W Ç Z } Z u } o } P Ç Æ u ] v ] } v ( } W Ç Z } o } P ] ~W W · Æ u ] v ] } v ( } W Ç Z } o } P ] ~W W E / d , E KK< í ^WW t } ] ] } v } ( ^ v W } À ] v ] o W Ç Z } o } P

v o Ç Z } u u v ] } v W hz

t r s€¦ · E Z u o P º Z W ï U ì ì ¦ µ P ^ v µ u u v W u Z v v P X õ W ï ì h Z u ] o t µ v P W ] < o v r µ v ' u Á µ v P ( } o P v Z ] ] } v ] v

T W Z U V Y X b a c Y W X U Z V W V U V U Z Y W Y X Z W U V W Z U V Y X W X Y U V …123userdocs.s3-website-eu-west-1.amazonaws.com/d/0f/37/... · 2018-01-18 · c _ v [d w ^ v u