1 idgf international desktop grid federation porting application to the boinc desktop grid: isa ras...
TRANSCRIPT
![Page 1: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/1.jpg)
1
IDGFIDGF
IInternationalnternational DDesktopesktop GGridrid FFederationederation
PORTING APPLICATION TO THE PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS BOINC DESKTOP GRID: ISA RAS
EXPERIENCEEXPERIENCE
Mikhail PosypkinMikhail Posypkin
Oleg ZaikinOleg Zaikin
Nikolay KhrapovNikolay Khrapov
DEGISCO is supported by the FP7 Capacities Programme under contract nr RI-261561.DEGISCO is supported by the FP7 Capacities Programme under contract nr RI-261561.
![Page 2: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/2.jpg)
2DEGISCO WP4 21/04/23 2
TALK OUTLINETALK OUTLINE
Introduction to the Russian Chapter Our BOINC and combined infrastructures OPTIMA@home project: using GenWrapper SAT@home project: using DC-API Performance Assessment Lessons learnt
![Page 3: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/3.jpg)
3DEGISCO WP4 21/04/23 3
Who we areWho we are
Institute for Systems Analysis of RAS (ISA RAS) www.isa.ru (founded in 1976) Centre for Grid Technologies and Distributed Computing:
Desktop and Service Grids Service-oriented Science Parallel algorithms Numerical methods in optimization
Partner in the DEGISCO project Founding member of IDGF
![Page 4: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/4.jpg)
4DEGISCO WP4 21/04/23 4
The Goals of Russian ChapterThe Goals of Russian Chapter
Help Russian research and industry teams to develop, deploy and maintain their desktop-grid distributed applications
Maintain DCI infrastructures for our projects and for other organizations
Provide courses and tutorials on desktop grids and volunteer computing for Russian academic and industrial community
Promote Russian BOINC projects Explain the mission of IDGF and benefits of the
membership
![Page 5: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/5.jpg)
5DEGISCO WP4 21/04/23 5
Our Desktop Grid InfrastructureOur Desktop Grid Infrastructure
BOINC Educational Infrastructureboinc-edu.isa.ru
BOINC Test Infrastructureboinc-test.isa.ru
BOINC Production Infrastructureboinc.isa.ru,sat.isa.ru
Based on SZTAKI Desktop Grid Package
![Page 6: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/6.jpg)
6DEGISCO WP4 21/04/23 6
The Typical PassThe Typical Pass
Netmax OPTIMA@homeSAT@home
Lots of (student) projects
![Page 7: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/7.jpg)
7DEGISCO WP4 21/04/23 7
Production BOINC InfrastructureProduction BOINC Infrastructure
Currently we have two projects running in production: OPTIMA@home (0.5 Tflops real performance)and SAT@home (1.3 Tflops real performance)
Two ways of application porting: – GenWrapper (OPTIMA@home)– DC-API (SAT@home)
= 3 x 48-core servers
![Page 8: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/8.jpg)
8DEGISCO WP4 21/04/23 8
Service Grid InfrastructureService Grid Infrastructure
4 nodes: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 4 GB RAM,640 GB Disk space
RU-ISA-CGTDC : grid.isa.ru
![Page 9: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/9.jpg)
9DEGISCO WP4 21/04/23 9
Combined InfrastructureCombined Infrastructure
EGI Desktop Grid VO
OPTIMA@HOME
![Page 10: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/10.jpg)
![Page 11: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/11.jpg)
11
OPTIMA@homeOPTIMA@home
•OPTIMA@HOME is a research project that uses Internet-connected computers to solve challenging large-scale optimization problems. The goal of optimization is to find a minimum (or maximum) for a given function.
•Currently we use Monotonic Basin Hopping Method implemented in the BNB-Solver framework
![Page 12: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/12.jpg)
12
Monotonic Basin-Hopping MethodMonotonic Basin-Hopping Method
Maximal No Improve (MNI)
MNI controls the run-time
“seed”- starting point
![Page 13: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/13.jpg)
13
Distributed Version of Basin-HoppingDistributed Version of Basin-Hopping
Generate Seeds
ComputeResults
Collect and ProcessResults (find minimum)
![Page 14: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/14.jpg)
14
ImplementationImplementation
BOINC CoreClient
GenWrapper
BNB-Solver
BOINC Server
Server Workstation
Client Workstation
![Page 15: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/15.jpg)
15DEGISCO WP4 21/04/23 15
GenWrapper- A generic BOINC GenWrapper- A generic BOINC wrapper for legacy applicationswrapper for legacy applications
Free tool developed by SZTAKIhttp://genwrapper.sourceforge.net/
GenWrapper is a generic BOINC wrapper for legacy applications utilizing GitBox (a variant of BusyBox): Use POSIX like shell scripting and built-in commands like tar, awk, sed, zip, etc. to control and execute your legacy Desktop Grid application.
![Page 16: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/16.jpg)
16
GenWrapper script for GenWrapper script for OPTIMA@home (on the client side)OPTIMA@home (on the client side)
IN=`boinc resolve_filename in`
OUT=`boinc resolve_filename out`
BNBAPP=`boinc resolve_filename bnbss`
./"${BNBAPP}" "${IN}" "${OUT}"
![Page 17: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/17.jpg)
17DEGISCO WP4 21/04/23 17
On the server sideOn the server side
Generate and store input files Use create_work to submit workunits Use sample_assimilator (shipped with BOINC)
to collect results in a folder Use scripts to process results
![Page 18: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/18.jpg)
18
More complex scenarioMore complex scenario
Generate Starting Points
ComputeResults
CollectResults and generate new workunits
![Page 19: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/19.jpg)
19
More elaborate versionMore elaborate version
We use a perl script that • runs as a periodic task• processes results , generates and submits new workunits• uses found local minima as starting points for new workunits
Adding periodic task (in config.xml):<task> <cmd> ma.pl </cmd> <period> 30 minutes </period> <output> ma.out </output> </task>
![Page 20: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/20.jpg)
20DEGISCO WP4 21/04/23 20
Resume: GenWrapperResume: GenWrapper
GenWrapper is perfect for porting “legacy” applications
You can organize complex workflows with GenWrapper using BOINC periodic tasks
If you need more control over your application use DC-API
![Page 21: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/21.jpg)
![Page 22: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/22.jpg)
22
SAT@home projectSAT@home project
SAT@home is a research project that uses volunteer computing to solve hard and practically important problems that can be effectively reduced to Boolean satisfiability problem.
Applicability• discrete functions inversion problems• cryptography• discrete optimization • bioinformatics
Problems that being solved currently:Problems of inversion of some cryptographic functions used in stream ciphers are being solved.
![Page 23: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/23.jpg)
23
The SAT@home approachThe SAT@home approach
Desktop grid decisionSAT
problem
CNF in DIMACS format
decompose
solve Satisfying assignment or UNSAT
CNF encoding of the original problem is decomposed into a family of CNFs which are "simpler" for SAT solvers. For finding "good" parameters of decomposition we use special technique of predictive functions which take into account peculiarities of the original problems.
Originalproblem
![Page 24: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/24.jpg)
24
SAT@home ImplementationSAT@home Implementation
Project was implemented using BOINC platform and DC-API library (http://www.desktopgrid.hu).Server part of the application terminates the experiment as soon as the 1st satisfying assignment foundClient part of the application is based on publically available SAT solvers minisat 1.14.1 and minisat 2.0 (http://minisat.se) modified to take into account peculiarities of an original problem and its CNF encodingControl points
![Page 25: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/25.jpg)
25DEGISCO WP4 21/04/23 25
DC-APIDC-API
DC-API applications consist of two major components: a master application and one or more client applications.
The master application usually runs as a daemon, but it is also possible to write a master that runs periodically. Master generates workunits, collects and processes results.
Client applications are simple sequential programs that take their input from the master, perform some computation on it and produce some output.
Free: http://www.desktopgrid.hu
![Page 26: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/26.jpg)
26DEGISCO WP4 21/04/23 26
Resume: DC-APIResume: DC-API
Flexible C-based Gives comprehensive control over your
application Much easier than BOINC API
![Page 27: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/27.jpg)
27
GenWrapper vs DC-API: when to use GenWrapper vs DC-API: when to use what?what?
Gen Wrapper• You have a big complex
application • You have a non-C (C ++)
application• You don’t need a fine
control over your distributed application
DC-API• You write application from
scratch• Your application is written in
C/C++• You need complex
distributed scenario and fine control over your application
• You target several DG platforms: OurGrid, Condor, etc.
![Page 28: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/28.jpg)
28
Performance evaluation: SZTAKI toolsPerformance evaluation: SZTAKI tools
![Page 29: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/29.jpg)
29
Performance evaluation: ISA RAS toolsPerformance evaluation: ISA RAS tools
![Page 30: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/30.jpg)
30
Performance evaluation: ISA RAS toolsPerformance evaluation: ISA RAS tools
Tuesday, November 8, 2011Session S8 16:15 Assessing the Performance of Desktop Grid Applications, A. Afanasiev, N. Khrapov, and M. Posypkin
![Page 31: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/31.jpg)
31DEGISCO WP4 21/04/23 31
Lessons LearntLessons Learnt
Choose proper technology (GenWrapper or DC-API)
Invest to advertisement (forums, statistics etc.)
Evaluate and analyze your performance (use tools)
Be careful with your donors – always respond their questions and don’t ignore their complaints
![Page 32: 1 IDGF International Desktop Grid Federation PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS EXPERIENCE Mikhail Posypkin Oleg Zaikin Nikolay Khrapov](https://reader036.vdocument.in/reader036/viewer/2022062423/5697bfec1a28abf838cb82ea/html5/thumbnails/32.jpg)
32
http://desktopgridfederation.org/http://desktopgridfederation.org/
Globe adapted from http://upload.wikimedia.org/wikipedia/commons/f/fa/Globe.svg
Globe adapted from http://upload.wikimedia.org/wikipedia/commons/f/fa/Globe.svg
IDGFIDGF
IInternationalnternational DDesktopesktop GGridrid FFederationederation