introduction to the boinc software
DESCRIPTION
David P. Anderson Space Sciences Laboratory University of California, Berkeley. Introduction to the BOINC software. Outline. Abstractions The BOINC server software The BOINC client software and runtime system. Design goals. A few applications, lots of jobs High performance - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/1.jpg)
Introduction to the BOINC software
David P. Anderson
Space Sciences LaboratoryUniversity of California, Berkeley
![Page 2: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/2.jpg)
Outline
• Abstractions
• The BOINC server software
• The BOINC client software and runtime system
![Page 3: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/3.jpg)
Design goals
• A few applications, lots of jobs
• High performance
– millions of jobs per day
• Scalability
• Fault tolerance
![Page 4: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/4.jpg)
Abstractions• Platform
• App version
– a collection of files, one of which is an executable main program
– associated with a platform
• App
– a set of app versions that all perform roughly the same computation
– may have versions for different platforms
– may have different versions for one platform (GPU, non-GPU)
![Page 5: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/5.jpg)
Abstractions
• Workunit (job)
– a collection of input files
– associated with an app (not an app version!)
– attributes• resource estimates and bounds
• latency bound
• Result (job instance)
– a collection of output files
– associated with a workunit
![Page 6: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/6.jpg)
Anatomy of a BOINC project
MySQLdatabase
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
serversdaemons andperiodic tasks
clients
![Page 7: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/7.jpg)
Work generator
MySQLdatabase
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
work generator
• Creates input files
• Creates workunits
• One per app
• Flow control
– disk space
– DB size
![Page 8: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/8.jpg)
Specifying a job
• Workunit template
– XML document describing• input files (logical, physical names)
• job attributes
• Result template
– XML document describing output files
• create_work()
– specifies templates, app, input files
![Page 9: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/9.jpg)
Validator
• Check result validity
• Compare replicas
• May be app-specific
MySQLdatabase
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
validator
![Page 10: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/10.jpg)
Validation
• Clients may
– return bad results
– exaggerated claimed credit
• Strategies
– app-specific consistency checking
– replication• fuzzy comparison
• homogeneous redundancy
– adaptive replication
![Page 11: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/11.jpg)
Assimilator
• Processes completed results
• App-specificMySQL
database
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
assimilator
![Page 12: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/12.jpg)
Summary
• Create app, app versions for different platforms
• Develop work generator
• Develop validator
• Develop assimilator
Isn’t there a simpler way?
![Page 13: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/13.jpg)
Single-job submission
• Assemble your input files and executable, thenboinc_submit --input foo --output blah program
• How this works:
– uses “wrapper” app
– executable is part of workunit
– templates are created automatically
• What it doesn’t do:
– multi-platform
– validation
![Page 14: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/14.jpg)
Job dispatch
MySQLdatabase
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
transitioner
feeder
scheduler(CGI or FastCGI)
share-memoryjob cache
clients
![Page 15: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/15.jpg)
File transfer
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
Apache
file uploadhandler
clients
![Page 16: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/16.jpg)
Janitorial daemons
MySQLdatabase
project root/bin/cgi-bin/download/
00/ .. 3ff/html/log_*/templates/upload/
00/ .. 3ff/
file deleter
DB purger
![Page 17: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/17.jpg)
Ways to deploy a BOINC server
• Linux server
• Server VM for VMWare
• Server VM for Amazon EC2
![Page 18: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/18.jpg)
The BOINC runtime system
Directory structure:
BOINC/projects/
lhcathome/physical_name0physical_name1
setiathome/slots/
0/logical_name0 (link file)logical_name1
1/
BOINC client
application
share-memorymessage-passing
BOINC runtime
fraction doneCPU time
suspendresumequit
![Page 19: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/19.jpg)
Basic API
• boinc_init()
– creates a thread that handles messages
• boinc_finish()
– creates a “finish file”
• boinc_resolve_filename()
– maps logical to physical file names
![Page 20: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/20.jpg)
Checkpointing
• boinc_time_to_checkpoint()
– call at points where you can checkpoint
• boinc_checkpoint_done()
– call when you’re finished checkpointing
![Page 21: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/21.jpg)
Compound applications
• Examples:
– coordinator program runs several worker programs in sequence
– “switcher” program probes CPU architecture, selects which executable to run
• Variants of boinc_init() let you specify which app is main program, and how messages are handled
• Each message type must be handled by 1 process
![Page 22: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/22.jpg)
Long-running applications
• Trickle-up messages
• Trickle-down messages
• Intermediate file transfers
![Page 23: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/23.jpg)
Legacy applications
• The BOINC wrapper
– takes XML “job file”
– handles all messages
![Page 24: Introduction to the BOINC software](https://reader035.vdocument.in/reader035/viewer/2022062517/56813c49550346895da5c636/html5/thumbnails/24.jpg)
GPU and multithread apps
• Server
– you supply a function that takes an app version and a host, and returns resource usage and estimated FLOPS
– the BOINC scheduler chooses the best version
• Client
– senses and reports coprocessors (e.g. NVIDIA GPUs)
– coprocessor-aware scheduling and work fetch