T: +27(0)51 401 9111 | info@ufs.ac.za | www.ufs.ac.za

eResearch at the UFS:

A Decade of Technology (E)nhancing Research

Mr. A. van Eck

MIN MIND

OUTLINE

• Looking at the Numbers:

– Research Groups

– Usage Statistics

– A Possible Alternative - Outsourcing

– Outputs

– Hypothetical ROI

• Looking at the Research

– Computational Chemistry

– Microbiology

– Medical Physics

RESEARCH GROUPS

Department # Of Researchers

Chemistry 5

Computer Sciences 3

Mathematics & Statistics 2

Medical Physics 4

Microbiology 7

Physics 4

Inactive Users (Not publishing) 16

Total Users 77

STATISTICS UP TO 31 OCTOBER 2014

Indicator Value

Number of jobs (2014) 43 348

CPU Hours (2014) 468y 348d 1:42

Running Jobs 215

Queued Jobs 337

Storage Used (Homes) 23.268 Tb

Storage Used (Scratch) 7.7033 Tb

Total CPU Hours since 2010 1983y 236d 6:06

STATISTICS USAGE PER USER

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

1,600,000

bar

nar

de

pb

rin

kaco

etz

eesj

con

rad

jd

uto

itp

dh

illtn

jon

asso

nk

kuh

nam

pre

tori

usc

van

eck

ave

nte

rgjs

bar

nar

de

pb

rin

kaco

nra

dj

du

toit

pd

enge

lbre

chti

hill

tnla

tega

nd

up

ree

zlm

iller

bva

ne

cka

van

soel

en

bve

nte

rgjs

con

rad

jd

uto

itp

dfr

eit

agr

mar

itzj

msc

hab

ort

dw

pvd

mer

we

kb

este

rpa

con

rad

jd

uto

itp

den

gelb

rech

tifr

eit

agr

kuo

ere

nyu

anl

stey

nm

trai

n0

8tr

ain

28

van

eck

avd

mer

we

kb

este

rpa

bo

tmae

caso

ned

con

rad

md

uto

itp

dfe

rrei

rah

fre

itag

rla

tega

nd

up

ree

zlm

aree

jo

ps0

30

pre

tori

usc

rie

kert

sjp

kst

eyn

mtr

ain

00

trai

n0

2tr

ain

07

trai

n2

3tr

ain

27

van

de

rme

rwe

sva

ne

cka

vdm

erw

ek

CP

U H

ou

rs

CPU Hours per User2010-2014

STATISTICS CPU HOURS PER MONTH

0

100000

200000

300000

400000

500000

600000

700000

800000Ja

n-1

0

Ap

r-1

0

Jul-

10

Oct

-10

Jan

-11

Ap

r-1

1

Jul-

11

Oct

-11

Jan

-12

Ap

r-1

2

Jul-

12

Oct

-12

Jan

-13

Ap

r-1

3

Jul-

13

Oct

-13

Jan

-14

Ap

r-1

4

Jul-

14

Oct

-14

STATISTICS CPU HOURS

2010 2011 2012 2013 2014 Average

January 58 901.10 327 930.60 190 923.90 282 451.70 398 049.50 251 651.36

February 0.00 305 592.00 241 200.20 349 951.40 365 862.90 252 521.30

March 77 747.30 314 536.20 376 733.50 537 793.50 501 068.60 361 575.82

April 85 886.10 92 072.50 427 189.40 484 007.80 314 049.20 280 641.00

May 73 313.90 349 264.30 569 267.40 672 655.10 432 609.40 419 422.02

June 46 557.80 451 322.80 420 690.00 136 617.50 582 281.80 327 493.98

July 20 505.30 507 175.40 216 653.20 0.00 320 829.30 213 032.64

August 17 063.80 403 608.50 47 968.50 602 980.70 293 276.20 272 979.54

September 47 568.70 426 124.70 54 066.90 550 190.50 280 724.00 271 734.96

October 71 075.90 425 248.30 307 924.90 465 896.10 581 916.40 370 412.32

November 114 557.90 430 309.50 0.00 358 190.80 37 366.40 188 084.92

December 36 235.30 269 688.20 596 422.90 426 654.40 0.00 265 800.16

649 413.10 4 302 873.00 3 449 040.80 4 867 389.50 4 108 033.70

AMAZON COST (US $)

2010 2011 2012 2013 2014 Average

CPU Hour 0.361 0.361 0.361 0.361 0.361 0.361

Instance/3yr

7 670.000 7 670.000 7 670.000 7 670.000 7 670.000 7 670.000

Transfer/Gb 0.120 0.120 0.120 0.120 0.120 0.120

Storage/Gb 0.125 0.125 0.125 0.125 0.125 0.125

Monitoring/Inst/Mnth

3.500 3.500 3.500 3.500 3.500 3.500

Discount > 250k

10.00% 10.00% 10.00% 10.00% 5.00% 9.0%

Discount 250k-2M

20.00% 20.00% 20.00% 20.00% 10.00% 18.0%

AMAZON COST FOR CPU HOURS ONLY (US $)

2010 2011 2012 2013 2014 Average

January 21 263.30 118 382.95 68 923.53 101 965.06 143 695.87 90 846.14

February 0.00 110 318.71 87 073.27 126 332.46 132 076.51 91 160.19

March 28 066.78 113 547.57 136 000.79 194 143.45 180 885.76 130 528.87

April 31 004.88 33 238.17 154 215.37 174 726.82 113 371.76 101 311.40

May 26 466.32 126 084.41 205 505.53 242 828.49 156 171.99 151 411.35

June 16 807.37 162 927.53 151 869.09 49 318.92 210 203.73 118 225.33

July 7 402.41 183 090.32 78 211.81 0.00 115 819.38 76 904.78

August 6 160.03 145 702.67 17 316.63 217 676.03 105 872.71 98 545.61

September 17 172.30 153 831.02 19 518.15 198 618.77 101 341.36 98 096.32

October 25 658.40 153 514.64 111 160.89 168 188.49 210 071.82 133 718.85

November 41 355.40 155 341.73 0.00 129 306.88 13 489.27 67 898.66

December 13 080.94 97 357.44 215 308.67 154 022.24 0.00 95 953.86

234 438.13 1 553 337.15 1 245 103.73 1 757 127.61 1 483 000.17

AMAZON TOTAL PROJECTED COST

2010 2011 2012 2013 2014 Average

CPU Hours Cost 234 438.13 1 553 337.15 1 245 103.73 1 757 127.61 1 483 000.17 1 254 601.36

Instances 25 566.67 115 050.00 132 946.67 150 843.33 150 843.33 115 050.00

Transfer/GB 144.00 144.00 720.00 720.00 76 320.00 15 609.60

Storage 1 500.00 4 500.00 6 000.00 7 500.00 10 534.21 6 006.84

Monitor 420.00 1 890.00 2 184.00 2 478.00 2 478.00 1 890.00

Discount 26 206.88 167 492.12 138 695.44 191 866.89 172 317.57 139 315.78

Total (US $) 235 861.92 1 507 429.04 1 248 258.96 1 726 802.05 1 550 858.14 1 253 842.02

Total (ZAR) 1 732 755.83 10 960 867.01 10 263 085.27 16 638 488.90 16 668 668.28 11 252 773.06

PUBLICATIONS 2013/14*

Description 2013 2014*

Publication (ISI, IBSS & SA DHET Accredited Journal) 16 13

PhD 0 12

MSc 3 2

Conferences (Talk) 5 8

Conferences (Poster) 7 13

Other – Lectures, assistance 3 6

* 2014 and ongoing

PUBLICATIONS 2006 TO 2014*

2006 2007 2008 2009 2010 2011 2012 2013 2014 On-going Total

Publications 7 7 7 9 14 18 9 16 12 1 100

MSc 1 0 0 0 0 1 0 3 1 1 7

PhD 0 0 0 2 1 3 0 0 0 12 18

Conferences 7 3 4 8 11 16 6 12 21 88

Posters 5 0 2 2 7 4 2 7 13 42

Other (lectures etc.) 0 0 0 2 2 0 0 3 6 13

Publications Total Grants to UFS – HPC Users

2010 2011 2012 2013 2014 On-going

Publications1 237 850.67 1 846 850.24 1 054 296.50 2 042 208.00 1 431 974.52 0.00

MSc0.00 197 518.17 0.00 717 412.50 233 644.71 233 002.50

PhD380 359.35 1 303 086.62 0.00 0.00 0.00 6 108 540.00

Total 1 618 210.02 3 347 455.03 1 054 296.50 2 759 620.50 1 665 619.23 6 341 542.50

Publications Total Grants to UFS – HPC Users

2010 2011 2012 2013 2014 On-going

Publications771 298.86 1 032 184.08 386 575.38 867 938.40 403 737.26 0.00

MSc0.00 71 741.25 0.00 209 702.25 116 988.75 116 988.75

PhD392 648.79 691 054.00 0.00 0.00 0.00 2 303 865.00

Total1 163 947.65 1 794 979.33 386 575.38 1 077 640.65 520 726.01 2 420 853.75

71.93% 53.62% 36.67% 39.05% 31.26% 38.17%

RETURN ON INVESTMENT

2010 2011 2012 2013 2014

Publications 771 298.86 1 032 184.08 386 575.38 867 938.40 403 737.26

MSc 0.00 71 741.25 0.00 209 702.25 116 988.75

PhD 392 648.79 691 054.00 0.00 0.00 0.00

SLE - 560 000.00 - 810 000.00 -1 220 400.00 -1 318 032.00 -1 423 474.56

Procurement -2 926 045.98 - 205 200.00 -1 122 838.44 0.00 - 56 048.31

Pot. Savings* 1 732 755.83 10 960 867.01 10 263 085.27 16 638 488.90 16 668 668.28

ROI - 589 342.50 11 740 646.34 8 306 422.22 16 398 097.55 15 709 871.42

CASE STUDY 01: CHEMISTRY

Understanding and interpreting a series of complexes and

possibly finding any trends that might exist.

• Why use a HPC

– Some complexes are not physically possible to

synthesize in a lab and is based on quantum chemistry

– Cost of transition metals:

• RhCl3 = R 3 000.00/g

• Platinum (Pt), Rhenium (Re), Rhodium (Rh), Gold

(Au), Palladium (Pd) etc.

– Radio Active Materials

• Technetium (Tc)

Case Study: Chemistry … Continued

Description Value

Number of Jobs 366+

Duration Per Job 300:00:00 to 1000:00:00

Total CPU Time 91y 331d 05:58

Speculated additional time for real world experimentation

5 to 6 years

* Real word experimentation not possible due to fact that some complexes

can not be synthesized and the fact that the Radio Activity Lab was not

available during the research

CASE STUDY: MICROBIOLOGY (THE FAR PAST)

• In 2006 a researcher was limited to analyze an Arc

Repressor Protein with:

– 1 136 Atoms

– 42 546 Solvent Atoms

– Time Scale of 2 to 7 Nano Seconds

• The simulation ran for a number of months on a desktop

computer

CASE STUDY: MICROBIOLOGY (THE PAST)

Histone H3 Tail

686 atoms + 25 086 solvent atoms

Simulation Time Scale: 500ns

Real-Time on PC (2011): ~ 3 months

Real-Time on HPC (2014): ~2 Weeks

CASE STUDY: MICROBIOLOGY (THE PRESENT)

Nucleosome core particle

25 086 atoms + 232 827 solvent atoms

Simulation Time Scale: 500ns

Real-Time on PC (2011): You must be joking

Real-Time on HPC (2014): ~2 Months

CASE STUDY: MICROBIOLOGY (THE PRESENT)

Dinucleosome

53 544 atoms + 594 576 solvent atoms

Simulation Time Scale: 500ns

Real-Time on PC (2011): Are you drunk?

Real-Time on HPC (2014): Feasible

CASE STUDY: MEDICAL PHYSICS

• SANReN Fibre M. Ring installed November 2014:

– UFS South Campus

– UFS Main Campus

– UNISA – Bloemfontein

– CUT

– Pelenomi Hospital

– Psychiatric Hospital

– National Hospital

– Mangaung Municipal Offices

CASE STUDY: MEDICAL PHYSICS

Video Time

FINDINGS

• Number of researchers increases over the years

– Next generation sequence scanners

– HPC forms part of academic curriculum & training

– HPC will expand further in other areas in near future

• Over R1m is generated in publications per annum

• Over 54% of content is generated on HPC

• Over R11m is saved by using HPC and not outsourcing

CONCLUSION

• Some research is impossible without HPC

– Quantum Chemistry

– Microbiology

– Medical Physics – High Radiation Levels in experiments

• The financial implication

– HPC is sustainable

– In this document the following costs were excluded

• Training, experiment equipment & materials

• Electricity & Air-conditioning usage

• External funding received

“When planning for a year, plant corn.

When planning for a decade, plant trees.

When planning for for life, educate people.”

- Chinese Proverb

…….Thank you very much for your attention