accelerating your research with microsoft azure
TRANSCRIPT
šš·š£
š·š”= āš»š + š» ā šÆ + š
Data
Acquisition & modelling
Collaboration and
visualisation
Analysis & data mining
Dissemination & sharing
Archiving and preserving
fourthparadigm.org
Data-intensive Research
X-Info
ā¢ Data ingest
ā¢ Managing a petabyte
ā¢ Common schema
ā¢ How to organize it
ā¢ How to reorganize it
ā¢ How to share with others
ā¢ Query and Vis tools
ā¢ Building and executing models
ā¢ Integrating data and Literature
ā¢ Documenting experiments
ā¢ Curation and long-term
preservation
The Generic Problems
Experiments &Instruments
Simulations
Literature
Other Archives
facts
facts
facts
facts
Questions
Answers
A-series
ā¢ 1-16 cores
ā¢ 0.75-112GB RAM
ā¢ 20-605 GB HDD
ā¢ Up to InfiniBand 40Gbit/s
RDMA network (MPI)
D-series
ā¢ 1-16 cores
ā¢ 3.5-112 GB RAM
ā¢ Up to 800GB SSD
G-series
ā¢ 32 cores
ā¢ 468 GB RAM
ā¢ 6.5 TB SSD
Parker MacCready: Univ. of Washington
Rob Fatland:, Wenming Ye, Nels Oscar, Microsoft Research
Modeling Workflow
Forcing Data
Processed into
Standard Format
Output
MODELModel-specific
Forcing Files
Raw Forcing
Data
Observations
Processed into
Standard Format
SKILL TESTRaw Observational
Data
Skill
Result
ROMS
Cluster
200 cores
1 week/year
2 TB
per model year
Standard
Post Processing
1 week
LiveOcean: Hybrid Architecture
HPClinux 150 cores
ForecastNetCDF files
LiveOcean
Serverā¢ Post Processing
ā¢ Pre-make .png āviewsā
ā¢ Archive NetCDF files
ā¢ API for web sites
ā¢ Admin.js
ā¢ Client.jsBlob Storage:
Forecast Copy
Science UserpythonAzure Table:
Log Info
Admin
Website
Client Websitehttp://mappable.azurewebsites.
net/liveocean/
Rivers
USGS
Atmosphere
UW WRFOcean
HYCOM
James Williams, SLAC CIO
āThe Azure for Research programme has helped the Marine Institute and our research partners understand how cloud computing can be used to advance collaborative marine research including by making on-demand compute and advanced analytical data services much more easily available to virtual research teams.ā
Eoin OāGrady, Information Services and Development Manager, Marine Institute (Ireland)
British Library Labs cloud
analysis of digital catalogues,
including 19th Century books
scanned by Microsoft.@MechCuratorBot
mechanicalcurator.tumblr.com
RaaS
SaaS
PaaS
IaaS
Cloud Services
Research collaboration and data
lifecycle services
Data management, application
services, collaboration tools.
Programming abstractions,
database support, runtime
systems
Virtual machines, reliable
storage, provisioning tools,
network bandwidth
Research
Marketplace
Analytics services and expert
consulting
Domain specific applications
and data access
Advanced development tools
and libraries to SaaS
developers
Specially configured virtual
machine templates
Use laptops &
desktop computers
Overwhelmed by
data
Finding analysis
ever more difficult;
sharing even
harder
www.azure4research.com
Azure for Research Russia Special Awards
ā¢ 250,000 compute hours, 20TB storage,
machine learning, NoSQL and moreā¦
ā¢ Apply by 15 Augā15 at
http://aka.ms/azureresearchrussia
