enabling research success - university of saskatchewan

ANNUAL REPORT 2015-16

INFORMATION AND COMMUNICATIONS TECHNOLOGY

RESEARCH COMPUTINGEnabling Research Success

2 3

2015-2016ICT RESEARCH COMPUTING - Annual Report

Director’s MessageEnabling research success through professionalism, excellence, and passion.

Our MissionWe advance the University of Saskatchewan’s research mission

by supporting researchers with a team of highly skilled professionals

and a cohesive suite of research-focused IT services.

Our VisionWe will earn broad recognition as an essential partner

in support of the University’s digitally-enabled research.

I’m pleased to welcome you to this inaugural annual report for ICT Research Computing.

Beyond covering 2015/16 activity, the report introduces our team and the services we offer to the research community. Throughout, you’ll find stories of research success and our role in enabling that success.

More than just advanced software and large-scale infrastructure, Research Computing is a group of highly skilled professionals, dedicated to the realization of the university’s research goals. The team has grown this past

year, adding capacity in software development, research data management and support for the health sciences. Many existing team members have risen to the challenges of new or expanded roles.

Reflecting the team’s diversity of expertise, 2015/16 work has ranged from the creation of a new shared model of high performance computing, to tackling complex storage needs, to dramatic improvements in quantum computing simulation.

Of particular note is our leadership in developing a national research data management platform. Collaborating with partner institutions, under the stewardship of Compute Canada and the Canadian Association of Research Libraries, this platform will facilitate the discovery and sharing of data between researchers across Canada and around the world.

On the horizon for the coming year are new services, including electronic laboratory notebooks and secure data management for clinical research. Continued growth of the team will further our capabilities in supporting “big data”, including visualization, image processing, bioinformatics and phenomics.

I feel privileged to work with this dedicated, passionate team, as we continue to support and advance the success of research at the University of Saskatchewan.

Thank you for joining us, Chad CollerDirector, ICT Academic and Research Technology

Adam McKenzie BSc, MSc (Computer Science)

Research Data Management Analyst - Adam supports USask and Compute Canada

researchers on the management of their research data, including its preservation,

movement, and storage, and develops software for the NRDR project.

Sergiy Stepanenko BSc, MSc (Electrical Engineering)

Advanced Computing Coordinator / WestGrid Site Lead - Sergiy coordinates the

effort of Compute Canada funded staff at the U of S, manages services for Compute

Canada, and manages local advanced computing services.

Todd Trann BSc (Electrical Engineering), BSc (Computer Science)

Lead Software Developer NRDR - Todd coordinates software development effort

across Compute Canada and CARL on the National Research Data Repository (NRDR)

project.

Juan Carlos Zuniga-Anaya MSc (E. Engineering), PhD (Computer Science)

Advanced Research Computing Analyst - Juan supports USask and Compute Canada

researchers using HPC infrastructure, including scripting, algorithm design, parallel cod-

ing and porting, and collaborates and co-publishes with University researchers.

Mike Winter BSc, MSc (Computer Science)

Senior Software Developer NRDR - Mike develops software for the NRDR project,

including in-depth analysis and optimization of preservation pipeline software.

Keith Jeffrey MSc (Electrical Engineering)

Project Manager / Senior Analyst NRDR - Keith manages the NRDR project, liaises

with local and remote researchers and research centers, and provides advice on IT

policy and governance.

4 5


Who We AreEnabling research success by providing services and technologies backed by dedicated experts.

Jason Hlady BSc, MSc (Physical Chemistry)

Manager - Jason has worked in research computing for more than 15 years at the U of S

and WestGrid/Compute Canada. In addition to managing the University’s Research Com-

puting group, Jason is the RDM lead on Compute Canada’s Science Leadership Council.

Song Hu BSc, MSc (Electrical Engineering), PEng

Analyst - Song supports researchers using Linux across campus, and provides research

support in targeted areas, including Geological Sciences and Engineering.

John Costa BSc (Physics), MSc (Biomedical Engineering)

Senior Analyst - John consults with senior leadership in Health and Medicine, analyzes

and develops solutions related to health research, including in the areas of Personal Health

Information and HIPA compliance, and provides input on IT policy and governance.

Research Computing staff accelerate the University’s research goals through a direct support model. The

group is highly capable and well educated—there are 2 PhDs and 7 MScs in the group presently. They assist

researchers in securing research funding through consultation and strengthening of proposals. They train the

next generation of HQP, provide expertise, maintain services and infrastructure, assist researchers in the use of

services and infrastructure, and create new services.

Ian MacPhedran BSc, MSc, PhD (Structural Engineering), PEng

Senior Analyst - Ian provides guidance and support to researchers across the breadth of

IT services, including research software, operating specialized research-enabling services

(e.g. GIS, NVIVO, DataShare), and grant application consultation.

Parallel Computing RDM ModelingImage Processing 3D PrintingGPU

Signal ProcessingBig Data SimulationPhysicsAssembly Language

OptimizationCompilers Health CanadaSecurity

Logic

Kernels

Electronics

Engineering

CAD

Numerical Linear Algebra Artificial Intelligence

GISFEA Linux Data Mining Visualization DatabasesSoftware Engineering

Algorithms Bioinformatics ProgrammingEngineering3DScientific Computing

Helping you do more, faster, better.

AccomplishmentsEnabling research success by contributing facilities and expertise to over 25 journal publications and 5 conference presentations.

1 40

42

25

7

ExcellenceIn 2015, Jason Hlady received

the highly coveted Compute

Canada Award of Excellence for

his dedication and commitment to

Compute Canada’s mission.

RoboticsSong Hu used his robotics expertise

to good effect by fixing Grizzly, a

research robot in Dr. Reza Fotouhi’s

lab. After disassembly, diagnosis,

and reassembly, this mechanical

four-wheeled Grizzly is back from

hibernation.

PublicationsResearch Computing has supported

or contributed to over 25 research

publications and 5 conference

papers in 2015, up from 19 in 2014.

Health SecurityJohn Costa and ICT Security

completed a Personal Health

Information environmental scan

for administrative units to assess

the University’s risks. Results will

inform new policy. Seven campus

units were surveyed.

SupercomputingJuan Zuniga and Sergiy Stepanenko

enabled John Tse (Physics) to

use one of the world’s biggest

supercomputers. Hosted in China,

Tianhe-1 was once the second

fastest supercomputer in the world

(2010-11).

TrainingAn advanced computing training

course (Introduction to Scientific

Computing) delivered by Juan

Zuniga was sold out — twice. The 40

attendees included faculty, students

and staff.

7


6

ProfessionalismExcellencePassion

“...ICT support of my computer clusters has been invaluable. With-out their expert experience, it would not been able to keep the clusters working without major interruption. This is essential to my research.”

- Dr. John S. Tse, Professor, Department of Physics

“ Jason’s expertise and devotion were essential in being able to launch Globus so successfully as a national service in such a timely manner. ”- Lindsay Sill, Executive Director of WestGrid at Compute Canada

“ It's invaluable to have this expertise available to faculty within the [College of Medicine] because it not only expedites our productivity and accelerates our discovery rate, but ensures that our data (a.k.a, our blood, sweat and tears) remain secure from loss via hardware errors or external attacks.” - Julia Boughner, Professor, Department of Anatomy and Cell Biology

8 9


What Can We Do For You?Enabling research success by delivering research infrastructure, software and consulting services.

Research Data ManagementManaging data is vital to every research effort. A data management plan

(DMP)— a plan for curation, access, discoverability, and preservation — ensures

your most expensive asset is fully protected and compliant with TriCouncil

funding policy.

Visualization & Image ProcessingApplying techniques to see data in novel ways using sophisticated visualization software

and hardware. From 2D and 3D digital models to 3D printed models, our specialists

can help transform your data.

Data Collection ToolsREDCap is a sophisticated, secure web application for building and managing

online surveys and databases, quickly and securely. Specifically tailored to the

needs of clinical research studies, REDCap is Research Ethics approved, and

ready to use.

We help you do more, faster, better.

Advanced Research Computing (ARC)ICT provides infrastructure and support staff to solve problems that are too big to

tackle on your desktop computer. The High Performance Computing Research Facility

(HPCRF) is a modern data centre that hosts supercomputer clusters for researchers,

as well as ICT’s advanced research computing systems. The WestGrid / Compute

Canada staff in Research Computing can help you access Canadian ARC resources,

design computer systems for research projects, and use the University’s ARC systems

effectively.

Consulting and AdvisingResearch Computing’s breadth of expertise across many domains makes light work

of heavy problems. Our experts help researchers design research solutions and

sharpen their grant applications. From recommending computational approaches

to directly collaborating with faculty on projects and tough problems, we enable

research success by providing expertise, saving you time—time that you can spend

on research, rather than technology.

Custom SoftwareResearch relies heavily on software to solve difficult problems. While many software

programs exist, other problems remain unserviced by commercial products. Research

Computing provides guidance and analysis for the development of custom software

programs.

10 11


Dr. Artur Sowa

Artur Sowa and Juan Zuniga collaborated on

simulation of quantum computers. In 2015, Juan

designed and implemented novel algorithms that

improved the performance of a quantum annealing

simulation to world record levels. The initial simulation

algorithm could model an 8-qubit quantum computer

in a server’s main memory. Over the course of their

collaboration, Juan’s algorithm design improvements

made it so Artur could simulate a 38-qubit quantum

computer in about the same amount of space. Each

qubit represents a doubling of the compute space,

and so Juan’s algorithmic improvements represent

a billion-fold (!) performance improvement. The

original algorithm would have required a Zettabyte

of memory in one computer — one billion Terabytes,

about equal in size to ALL the traffic over the global

Internet in 2015. With Juan’s improvements, the

simulation was completed on a single computer at

the University this year!

A Billion Times Bigger

The cost of medical imaging depends crucially

on the trade-off between the signal acquisition

and processing time versus final image quality.

Any improvement in either the hard-wired signal

acquisition apparatus or the digital signal processing

algorithms translates directly into improved

diagnostics and reduced healthcare cost, e.g. in

recent years, significant progress was achieved via

an application of innovative compressed-sensing

techniques. Opportunities for further progress

are now emerging from new insights into signal

processing algorithms in conjunction with new

concepts for control of signal acquisition in a quantum

environments such as an MRI machine. Images to the

right contrast the effects of quantum denoising.

Quantum Image Processing

Consulting ComputingAwards and Grants

“ ...with Juan Carlos and others we have produced world-class results, challenging those of bigger, better funded teams. ”

- Artur Sowa, Professor, Department of Mathematics and Statistics

12 13


Research Computing deploys some of the

most powerful scientific software programs

in the world in for the research community.

The most popular titles include ArcGIS

(mapping and spatial analysis), MATLAB

(numerical analysis and simulation) and

Mathematica (support for symbolic algebra).

Over the course of the year, hundreds of

students and faculty use these tools tens

of thousands of times.

With ICT, the research community leverages

these powerful scientific tools to advance

their research.

The assortment of software tools and

technologies empower and enable

researchers to achieve results that, with

only a few hours, would rival the life-time

productivity of researchers only a decade

ago.

Research SoftwareEnabling research success by providing specialized research software programs and the expertise to support them.

44 instruments

187 studies

21 users

58,417 uses

37,142 uses

8,968 uses

7,375 uses

5,920 uses

Do more with research software. ICT hosts software services and manages the deployment and licensing of essential research software packages.

REDCapArcGIS

MatLabMathematica

OriginMaple

NVIVO is an analytical tool designed to support qualitative research. Available

to researchers at no charge, ICT works in partnership with the Social Sciences

Research Laboratories (SSRL) to support NVIVO. Together, we provide services to

help researchers. ICT Research Computing supports NVIVO server for collaborative

projects — researchers work on the same data through a shared NVIVIO database.

Scientific Software

ICT recently added ORIGIN, a leading package for producing publication-ready

graphics, to our library of scientific tools for researchers. Origin offers plotting

features unavailable elsewhere, including digitization of pictures of graphs—

converting scanned images and extracting the data. Origin was used over 1500

times in May alone. Origin and OriginPro are available to the research community

at the U of S and CLS.

Software: By The Numbers

14 15


Global Institute for Water Security

The Global Institute for Water Security (GIWS) has big

computing needs. Modeling water usage, weather

patterns, and climate takes serious horsepower. In

2015, ICT Research Computing and GIWS partnered

to develop a shared model of high performance

computing (HPC) that enables the GIWS to leverage

more computing power than they could purchase on

their own. The traditional way to use these servers

would be to purchase a rack to hold the computers,

storage, private network switches and cabling, and

get a system administrator to build and maintain a

standalone HPC cluster for the GIWS. A standalone

cluster wouldn’t allow GIWS researchers to use more

computers than they purchased, nor to share the

systems when they weren’t being used.

Instead, the GIWS bought 20 servers to add to

the Plato cluster, and used software to guarantee

GIWS researchers access to the same amount of

computational resources as a standalone system

would have provided. The whole is greater than the

sum of the parts. GIWS 1, Research Computing 1.

The Global Institute for Water Security

(GIWS) and its partners foster collaboration

to address regional and global challenges

of Water Security, including the sustainable

use and protection of water resources and

protection against water-related hazards

such as flooding and drought.

Key research themes include Climate

Change and Water Security. The Institute

works to ensure that society globally

has the understanding and the tools

to sustainably manage and protect the

world’s water resources, and that Canada

has the research and expertise needed

to understand and manage our water

systems in an era of rapid societal and

environmental change.

Ground Water Modeling

Win-Win Shared Computing

StorageConsultation

“ …ICT has been a great support for our researchers to help set up and run their analyses. ”

- Saman Razavi, Professor, SENS and GIWS

Computing

National Research Data Repository

16 17


Enabling research success by providing convenient access to secure, backed up storage for research data.

Storage Services

Research data is the most important asset of the research enterprise. Keeping it safe, yet convenient to access and share, is of paramount importance to ICT. The Cabinet service provides fully redundant and backed up storage right to the desktop. Designed for day-to-day access to frequently changing data, it is perfect for business and academic documents, and small research data that needs to be at your fingertips. For convenient and secure sharing of files with colleagues, Cabinet is integrated with OwnCloud, a “DropBox-like” service that runs at the University.For larger research data, ICT provides DataShare, a shared network disk that provides space for your research group.

DataShare permits large files to be easily shared among many users on campus, straight to the desktop. Over 100 groups use DataShare—with their data securely stored and backed up, researchers need not worry about disk failures, loss or theft. For the biggest of datasets, ranging from tens to thousands of terabytes, Research Computing has the experts who can help researchers access Compute Canada’s national storage platform, from getting resource allocations to moving big data across the country. Whether your data is measured in megabytes or in petabytes, Research Computing can help you keep your data safe, and share it with your collaborators.

Data of all shapes and sizes

Enabling research success by building a framework for preserving and discovering Canadian research data.

A team at the University of Saskatchewan in Research Computing is at the centre of a project to develop the technology for a national research data repository. With funding from both CFI MSI and the Province of Saskatchewan, Compute Canada and the Canadian Association of Research Libraries are collaborating with the U of S team on a two year project to build a scalable national platform for research data management and discovery. Tri-Council funding agencies state that “Research data resulting from agency funding should normally be preserved in a publicly accessible, secure and curated repository or other platform for discovery and reuse by others.” This includes expectations for researchers

to create data management plans, and to provide the necessary metadata to facilitate understanding and reuse. The pan-Canadian platform that we are building will provide tools and services to support the curation, access, discoverability, and preservation of research data, allowing researchers across Canada in a range of disciplines to have access to publicly funded data, addressing a longstanding gap in Canada’s digital research infrastructure. This service will also provide a framework that allows existing and future data repositories to be federated within a coherent system.

Pan-Canadian Platform

1716

18 19


Dr. Julia Boughner

Big problems require big solutions. Dr. Boughner’s

custom built Optical Projection Tomography (OPT)

machine is a purpose built system designed to capture

3D, high fidelity CT (computed tomography) images of

early stage embryonic development of various small

mammals.

The question for the Boughner lab is, how do you store

large amounts of data securely and reliably? The answer

to that question came from Research Computing.

Expecting to generate more than 10 TB of data over

the lifetime of the project, storage and the supports

were key.

Adding 15 TB of space to DataShare did the trick.

Now raw imaging data is seamlessly accessible from

the acquisition computer to the rendering computer,

without the need to manually transferring the data,

without the worry of data loss and with the comfort of

knowing support is just a phone call away.

My NSERC- and CFI-funded research

team is imaging, in 3D, the expression

patterns of genes that drive development

in the mouse face, with a focus on jaws

and teeth. Also using 3D image data

(micro-CT and OPT), we are deciphering

the developmental-genetics processes

that coordinate change among teeth as

well the bones and muscles of the head.

We use this new knowledge to clarify

evolutionary changes, and disease

etiologies, in primate including human

heads, jaws and dentitions.

Evolutionary developmental biology / anthropology

StorageConsulting

Big Storage For Big Data “ Having secure, efficient hardware and storage is a keystone of my research team’s productivity over the short- and long-term.”

“ I appreciated their consultation to identify my needs, as well as their advice about what would best advance my research approach. ”

- Julia Boughner, Professor, Department of Anatomy and Cell Biology

TFLOPS GB RAM

CORESNODES

USERS

Plato is an HPC cluster designed to support research projects, serving the entire University of Saskatchewan. Zeno

is a computing cluster with fast networking and specialized graphics processors to accelerate scientific computing.

Meton is a single computer with considerable CPU power and even more memory (RAM). These systems are

great for getting research done, and are the perfect place to develop projects before scaling up to more powerful

systems in Compute Canada. HPC Research Computing experts on campus are available to discuss your needs.

Compute Canada leads the acceleration of research and innovation by deploying state-of-the-art advanced research computing (ARC) systems, storage and software solutions. Research Computing can connect you to Compute Canada systems and experts.

20 21


Specifications

PLATO

Advanced Research Computing: By The NumbersEnabling research success by providing Plato, Zeno, and Meton through the High Performance Computing Research Facility.

18563712

214116

77

96,101 jobs

45 groups

214 researchers

1103 yearsof computing in one year

10.6 TFLOPS

8 Nodes

92 Cores

256 GB RAM

ZENO

Specifications

250 kWHPCRF delivers 250 kW of electricity. Enough to

power 9 homes.

Power

855xPLATO is 855 times as powerful as a modern

business computer.

Performance

10HPCRF hosts 10 clusters from various groups on

campus.

Hosting

METON1.6 TFLOPS

48 Cores

2048 GB RAM

1 Gb Network

Specifications

[email protected]

Enabling Research Success

enabling research success - university of saskatchewan

Documents