MuPIF Training

27.-28. March, 2017Czech Technical University, Faculty of Civil Engineering, Prague

AgendaMar 27

13:30-15:15 MuPIF: Introduction, design and installation (BP)

15:45-17:30 MuPIF: Network communication, examples (VS, ALL)

18:30 Dinner in “Bernard PUB Cesta Časem”, Na Kocínce 210/3

Mar 28

8:30-10:00 MuPIF: Coupled/linked models

10:30-12:15 MuPIF: API Development.

12:30-13:30 Lunch

13:30-15:15 Practical session, Discussion

2

Part 1: Introduction to MuPIF platform

Bořek Patzák, Vít Šmilauer, Martin Horák

Czech Technical University, Faculty of Civil Engineering, Prague

3

Content of Part 1• MuPIF Platform design – an overview• Distributed aspects• Platform installation• Basic platform operations, interactive session

– Connecting applications– Fields, field mapping– Visualization

4

MuPIF integrative role in Composelector

 

  

DEC

ISIO

N S

UP

PO

RT A

PP

SLC

E,

Ligh

t W

eig

hti

ng,

Vis

ualiz

ati

on

Cost

ing,

Res

Subs

Ass

ess

ment.

etc

…

DEC

ISIO

N S

UP

PO

RT A

PP

SLC

E,

Ligh

t W

eig

hti

ng,

Vis

ualiz

ati

on

Cost

ing,

Res

Subs

Ass

ess

ment.

etc

…

  

The simulation pipeline in MuPIF consists of top-level script in Python language linking and steering the individual models.

Distributed computing

Database

MaterialModel #3

MaterialModel #2,

Visualization

MaterialModel #5

MaterialModel #4

MuPIF● Steering scripts

for material and process modelling

● Database interrogation

● Integrates computational facilities

● Seamless data flow using VPN

● Material and process modelling

● Business model and optimization

SimulationChain

Validation

ProcessModel #1

BusinessModel,

Optimization

MaterialModel #1

Validation

Consider car as example• Complex system

• Need for simple interface hiding complex details allowing to operate the car

Introduction to OO design

Introduction to OO design

Consider car as example• Complex system

• Need for simple interface hiding complex details allowing to operate the car

• Different car types: need for standard interface allowing to drive all car types

Car Interface

Consider car as example• Object is encapsulation

of attributes and methods• Methods define interface

to interact with object• Classes can form

hierarchies to express similarity (inheritance)

CarEngineGearWheels[4]PlateID...

start()stop()turnLeft (angle)turnRight (angle)accelerate (rate)decelerate (rate)

Bus

NumberOfSeats

openDoor()closeDoor()

Introduction to OO design

10

Summary

• Objects represent abstraction of (any) entity in the modeling space. • Class is definition of object. Object is an instance of a class.• Object is encapsulation of data and methods • Objects can be manipulated and data exchanged using methods only• Classes can form hierarchical structure allowing to describe similarity or

a-kind-of relation (Inheritance) between classes• Abstract classes define public interface to be implemented by derived

classes

• Interface concept allows to hide internal data storage formats (natural supports for multiple data formats, design is data format independent)

Introduction to OO design

11

MuPIF simulation workflows

Design based on interacting, distributed objects (components)

• MuPIF is trying to make an abstraction of models and complex data to enable the idea of plug-and-play work-flows

• Models and complex data represented as software components, implementing common interfaces

• Top level classes for specific component group define common interface shared by all derived classes

• Python language extended by introduction of high level classes

MuPIF Platform – Design ideas

Advantages of design based on interacting objects (components)

• Internal data and storage formats naturally hidden behind common interface, native support for multiple data formats (objects manipulated using the same interface)

• Instead of trying to standardize data structures, MuPIF is focused on identification and standardization of methods on similar components (models, fields, microstructures,...)

• These methods are defined by interfaces defined by top-level classes and to be implemented by derived classes

•• Interface concept allows to manipulate similar components using same

generic interface – abstraction• Models will get data and operations on data in one consistent package,

do not have to interpret data themselves• Meta data can be attached to any component• Components in MuPIF can represent local as well as remote data, can be

distributed

MuPIF Platform - Design

Application

getField (id)getProperty (id)…

Internal Data

Interface

Field

evaluateAt(pos)getMesh ()…

Internal Data

Interface

Application Class representing an external application – Model or Postprocessor. Provides methods for getting and registering properties, fields, and functions, processing specific solution step, update state, termination, etc.Derived classes: • micress (acsrvappmic1.access.rwth-aachen.de),• mmpraytracer (mmpserver.erve.vtt.fi), • LAMPS

Field Field is a scalar, vector, or tensor quantity defined on a spatial domain (represented by the Mesh class).

Property Characteristic value of a problem, which has no spatial variation.

Function Function is an object defined by mathematical expression and can be a function of spatial position, time, and other variables.

Mesh Representation of a computational domain as a collection of Cells and VerticesDerived classes:• UnstructuredMesh

MuPIF Platform – Base classes

Vertex Represents Vertex.

Cell Interpolation Cell, whose geometry is defined using verticesDerived classes:2D: Triangle_2d_lin, Quad_2d_lin3D: Tetrahedron_3d_lin, Brick_3d_lin

TimeStep Represents solution time step.

Units Representation of units. Allows to convert units and check for unit compatibility.

MuPIF Platform – Base classes

Service Description__init__ (self, file) Constructor. Initializes the application.

ARGS:• file (str): path to application initialization file.

getField(self, fieldID, time) Returns the requested field at given time. Field is identified by fieldID.ARGS:• fieldID (FieldID): identifier• time (double): target timeReturns: Returns requested field (Field).

setField(self, field) Registers the given (remote) field in application.ARGS:• field (Field): remote field to be registered by the applicationReturns: None

getProperty(self, propID, time, objectID=0)

Returns property identified by its ID evaluated at given time.ARGS:• propID (PropertyID): property ID• time (double): time when property to be evaluated• objectID (int): identifies object/submesh on which property is evaluated

(optional)Returns: Returns representation of requested property (Property).

setProperty(self, property, objectID=0) Register given property in the applicationARGS:• property (Property): the property class• objectID (int): identifies object/submesh on which property is evaluated

(optional)Returns: None

Example: Application class interface

Service DescriptionsolveStep(self, tstep, stageID=0, runInBackground=False)

Solves the problem for a given time step. Evaluates the solution from actual state to given time. ARGS:• tstep(TimeStep): solution step• stageID(int): optional argument identifying solution stage• runInBackground(bool): if set to True, the solution will run in background (in

separate thread), if supported.

Returns: NonefinishStep(self, tstep) Called after a global convergence within a time step.

ARGS:• tstep(TimeStep): solution stepReturns: None

getCriticalTimeStep(self) Returns the actual (related to the current state) critical time step increment (double).Returns: critical time step (double)

getAssemblyTime(self, tstep) Returns the assembly time related to a given time step. The registered fields (inputs) should be evaluated in this time.ARGS:• tstep (TimeStep): solution stepReturns: assembly time (double)

terminate(self) Terminates the application.Returns: None

getAPIVersion(self) Returns the supported API version.Returns: API version (int)

Note: Full API specification part of public D1.1: Application Interface specification

Example: Application class interface

Native API implementation • Direct calls to application's

function• Python interface (python code,

wrapper-Swig, Boost)

Indirect API implementation• Wrapper around I/O streams or

files, more common

MuPIF MuPIF

Application 1

API

Application.lib

Application 1

API

Application.exe

Application interface implementation

ApplicationInit()solveAt (timeStep)getField (id)setField (field, id)

All components of the same type can be manipulate using the same interface defined by abstract class, internal details (data storage, functions) hidden

FieldevaluateAt (pos, time)

Simulation Scenario

CFD Celsian

Micress

FEM Field

HDF5 Field

Platform user Component developer

Key features of the platform

23

Transparent, distributed object system fully integrated into Python using so-called proxies.

A proxy is a special kind of object that acts as if it were the actual object, forwarding the calls to the remote objects, and pass the results back to the calling code.

Optional SSL based encryption for security

Local scenario

Distributed scenario

App1 App2

App1

App2

App2 Proxy

Name server

Pyro

Internet

Distributed environments

24

obj1 obj2

A proxy is local representation of remote object. • It intercepts the method calls you would normally do on

an object• transfers the call with arguments to the computer that

contains the remote object • transfers the results back to the caller

Calling code doesn’t have to know if it’s dealing with a normal or a remote object, because the code is identical.

obj1 obj2obj2 Proxy

Distributed environments, remote objects

Local/Distributed environmentsDistributed scenario Transparent, distributed object system

fully integrated into Python using so-called proxies.

Name server for resolving remote objects locations

Optional SSL based encryption for security

App1 App2 App1 App2App2 Proxy

Pyro name server

Pyro

Local scenario Communication between objects

by direct method calls

25

Communication with remote objects

• Pyro will automatically take care of any registered objects that you pass around through remote method calls, replace them by a proxy automatically.

• The receiving side obtains representation of remote object instead of a local copy.

• This feature can be enabled or disabled (AUTOPROXY config item)

26

Communication with large remote objects - possible scenarios

• Direct communication with remote object

• Direct aggregated communication

• Communication with local copy

ff Proxy

Pyro

c = []for p in points: c.append(p.getCoordinates())f.evaluateAt(coords)

fCopy of ffor p in points: f.evaluateAt(getCoordinates())

ff Proxy

Pyro

for p in points: f.evaluateAt(p.getCoordinates())

27

Communication with large remote objects - possible scenarios

Benchmark of remote Field Evaluation – 2D unstructured mesh (10k cells)– 1000 random field evaluations

Coping the remote field took 0.39 [s]

Evaluation method Time [s]

Separate remote field evaluation 7.52

Batched remote field evaluation 3.15

Separate local field evaluation 2.50 + 0.39

Batched local field evaluation 2.57 + 0.39

28

MuPIF installation

MuPIF Platform – how to access

• From the MuPIF pages (mupif.org)• Directly on SourceForge

http://sourceforge.net/projects/mupif/– Access/browse the code– Register for notifications/updates– Get support (Forum, Bug tracker)

• Getting the code directly via GITgit clone git://git.code.sf.net/p/mupif/code mupif.git

30

Setting up the platform

• General dependences:– Python (2.7.x recommended), – Python modules (Pyro4, pyvtk, NumPy)– ssh client (ssh, putty), ssh server

• Recommended: – Pip (tool for installing and managing Python

packages)– Paraview (vtk based visualization tool), – GIT client (distributed version control system)

31

Recommended setup on Windows

1. Install Anaconda scientific python package: https://store.continuum.io/cshop/anaconda/

2. Install mupif package

3. Recommended● VPN client or putty (ssh client, http://www.putty.org/)● Paraview (http://www.paraview.org/)

32

$ pip install mupif

Recommended setup on Unix/Linux

1. Install Python 2.x using your package manager or download installation package https://www.python.org/downloads/

2. Install pip (package manager or https://pip.pypa.io/en/latest/installing.html)

3. Install required python modules using pip:

4. Install VPN or ssh clients

33

$ pip install mupif

MuPIF directory structure

34

MuPIF_TOP_DIR - contains source code and other files of the MuPIF package +--mupif - contains source code of the MuPIF package | |--doc - documentation (reference manual and User guide) | |--examples - examples and tests | |--tests – unit tests | |--APIs – available application interfaces | |--Physics - module for units | |--tools - various supportive tools | |--*.py - MuPIF classes |--__init__.py - description of MuPIF module |--README.txt - general description |--setup.py - support for setuptools |--MANIFEST.in - support for setuptools

Typical MuPIF_TOP_DIR locations● C:\Users\user\Anaconda2\Lib\site-packages\mupif (Anaconda + pip install mupif)● /usr/local/lib/python2.7/dist-packages/mupif (Linux + pip install)

35

•Web interface http://sourceforge.net/projects/mupif/

MuPIF homepage – Git, Tickets

•Tickets – bug reporting, wishes

36

•mupif/examples/testExamples.sh – routine tests for Example01-Example07, Example09, Example10•Common configuration in mupif/examples/conf.py•These tests are routinely run after each commit on github.com•Test both for python 2.7.* and 3.0.*•Email to the developers if any problem

MuPIF routine tests

37

Overview of MuPIF tests

Local apps

Distributed apps

Direct access

Indirect access

Interpolation, VTK, serialization

Job Manager

Units

Example01

Example02

Example03

Example04

Example05

Example06

Example06-JobMan

Example07

Example09

Example10

• Standard tests with used features / extensions

MuPIF interactive session

Interactive session

• You can start interactive Python session and type commands directly

• Perhaps more convenient approach is to create a new simple script in editor (notepad++) and then run it using python

$ python>>>

$ edit demo.py…$ python demo.py

39

Python basics

• No need to declare variables

• Code blocks are defined by their indentation. "code block" means functions, if statements, for loops, while loops, and so forth. Indenting starts a block and unindenting ends it. There are no explicit braces, brackets, or keywords.

$ python>>> value=1.0

>>>for i in range(10):>>> print i>>>

40

Python basics

• To create an instance of a class (object):

• To invoke a method on object:

$ python>>> import sys # import sys module >>> from mupif import Vertex # import Vertex class>>> v = Vertex.Vertex (1, 1, (10., 10., 0.)) # create an instance of Vertex

>>> print v.getCoordinates()

Demo01.pyDemo01.py

Demo01.pyDemo01.py

41

• Simple example to illustrate Filed evaluateAt,

vtk export, and data exchange between apps.• Simple mesh generator on rectangular domain• Field mapping, result in vtk

42

Simple example – interactive session

Interactive sessionimport sys # import sys module to modify system pathsys.path.append('../..') # add path to mupif

from mupif import FieldID # include definition of some mupif classesfrom mupif import Fieldfrom mupif import ValueType

import demoapp # import a simple demo applicationimport meshgen # import a simple mesh demo module

43

Interactive sessionBasic field operations

app = demoapp.demoapp(None) # create instance of demo applicationprint app.getApplicationSignature() # print application signature

f = app.getField(FieldID.FID_Temperature, 0.0) # get sample field defined by demo appprint f.evaluate((5.0, 5.0, 0.0)) # evaluate field at given position

#evaluating field outside domain -> error#correct waytry: print f.evaluate((5.0, -5.0, 0.0)) #evaluate field outside domainexcept ValueError as e: # catch ValueError exception print "Warning: point is outside the domain“ print 0.0

#make some graphicsf.field2Image2D(title='Temperature', fileName='thermal.png')#export field to vtkf.field2VTKData().tofile('example1')

Demo02.pyDemo02.py

44

Interactive sessionSimple field mapping

#generate target mesh using meshgen: meshgen(origin, size, nx, ny)mesh2 = meshgen.meshgen ((0.0, 4.5), (10.0, 1.5), 50, 10);

values=[]; #list to store interpolated vertex valuesfor i in mesh2.vertices(): #loop over target mesh vertices val = f.evaluate(i.getCoordinates()) #evaluate source field in target mesh vertices values.append(val)

#create new field on target mesh using interpolated valuesf2=Field.Field(mesh2, FieldID.FID_Temperature, ValueType.Scalar, None, 0.0, values);#output vtk representation of mapped fieldf2.field2VTKData().tofile('example2')

Demo03.pyDemo03.py

45

Interactive sessionSimple field mapping – visualization

Visualize the vtk file(s) in paraview– File->Open, select vtk file– Filters->Alphabetical->Warp by scalar

46