the dawnsci eclipse science usecases jun aishima software engineer diamond light source didcot, uk
TRANSCRIPT
![Page 1: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/1.jpg)
The DAWNSci Eclipse science
usecases
Jun Aishima
Software Engineer
Diamond Light Source
Didcot, UK
![Page 2: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/2.jpg)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
![Page 3: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/3.jpg)
SynchrotronAKA – cool word, but what does it
mean? syn·chro·tron/ˈsiNGkrəˌträn/
Noun:A cyclotron in which the magnetic field strength increases with the energy of the particles to keep their orbital radius constant.
“They are machines which produce very strong light used for many different type of scientific experiments and sometimes other things.”
![Page 4: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/4.jpg)
Diamond Light Source, Didcot, Oxfordshire, UK
![Page 5: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/5.jpg)
• (Video has been removed to allow the talk to fit into size guidelines – see http://www.diamond.ac.uk/Public/Multimedia/Animations.html to view the animation)
![Page 6: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/6.jpg)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
![Page 7: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/7.jpg)
Detectors ofVarious flavors
![Page 8: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/8.jpg)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
![Page 9: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/9.jpg)
…180 images
The collected data may look like this…
![Page 10: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/10.jpg)
When what you really want is this…
![Page 11: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/11.jpg)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
![Page 12: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/12.jpg)
DAWN - Acquisition and Analysis with RCP
@ Diamond Light Source
Client server technology
Communication withEPICS and hardware
Scan mechanism
www.opengda.org
Jythonand Python
Visualisation
Communicationwith external
analysis
Analysistools
Data read, write,convert
Metadata structure
Workflows
www.dawnsci.org
DAWN is a collection of generic and
bespoke ‘views’ collated into
‘perspectives’.
The perspectives and views can be
used in part or whole in either
the GDA or DAWN.
Acquisition Analysis
![Page 13: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/13.jpg)
Collaborative Approach
Kichwa Coders Ltd.
Science Working Group
External Users
Consultants
Members
Now an Eclipse project!https://projects.eclipse.org/projects/technology.dawnsci
![Page 14: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/14.jpg)
‘Shoulders of Giants’
AKA – didn’t think of everything• RCP many of the core features, editors, toolbars, views, projects
• Ptolemy 2 (a version known as ‘Passerelle’) workflow and pipelining
• GEF for visualization of pipeline graphs
• Draw2D for 1D and 2D plotting (SWT XY Graph)
• Pydev for python/jython scripting layer used by the scientists
• HDF5 libraries for storing large data sets
• SWT/Jface – lazy viewers being used extensively for large trees and tables
• Apache, Eclipse-WST, springsource, JDK, and many more of course...
![Page 15: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/15.jpg)
![Page 16: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/16.jpg)
Did you see our poster yesterday, presented by Bahá?
• The DAWNSci Eclipse Project: How Eclipse Is Used for Science (Bahá El Kassaby)– Printed copies or PDF file available! Ask us!
![Page 17: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/17.jpg)
Data in DAWNSci
Example : Loading and Manipulating Data
This example• Columns of ASCII data• Separated by tabs• Read as 1D arrays
Other format with a loader: srs, dat, flt, gff, mca, csv, xy, xye, txt, tif, tiff, cbf, img, ciff, mccd, edf, pgm, cor, bruker, jpg, jpeg, png, f2d, msk, mib, mar3450, pck3450, raw, mrc, gz, bz2, zip, h5, hd5, hdf5, nxs, nexus, hdf, mat
Extension point to add more...
![Page 18: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/18.jpg)
Demonstration – Visual Tools
![Page 19: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/19.jpg)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
![Page 20: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/20.jpg)
The Processing pipeline
• Previous systems– Required much effort by the developer– User experience was too generic – not optimized for this
purpose
• Simplify (coding), simplify (GUI), simplify (capabilities)
• Result: wide adoption and rapid development for several experimental techniques
• Developed by Jacob Filik at Diamond• Available in DAWN
![Page 21: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/21.jpg)
Non-crystalline diffraction
Why Processing in DAWN?
![Page 22: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/22.jpg)
Simple? Operation
Calculation
Errorhandling
![Page 23: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/23.jpg)
Simple? Model
Allow drag and dropProvide a button to open file dialog box
Radio box in GUI
Number box in GUI with limits
Mouseover in GUI
![Page 24: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/24.jpg)
Simple?
File list
Input
Output
List of processing steps
Model
![Page 25: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/25.jpg)
Demonstration of pipeline
![Page 26: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/26.jpg)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
![Page 27: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/27.jpg)
Create your own operations and pipelines!
• Create an operation – org.eclipse.dawnsci.analysis.api.operation extension point– Dataset operations – including lazy operations
• Create a model – if necessary– Any additional data required for the calculation– Reflection on data type and annotations are used to
build the GUI for users to enter parameters
![Page 28: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/28.jpg)
A lot of existing, reusable operations
• Sector integration
– Typically, reduce a two-dimensional image into a one-dimensional dataset
• Normalization
– Remove frame-to-frame variation to put them all onto the same scale
• Subtraction
– Removing artifacts by subtracting background data from the sample data
• Averaging
– Multiple noisy points can be averaged
![Page 29: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/29.jpg)
Future work on the Processing pipeline
• Support more sciences (XPDF)
• Add branching, merging, and other non-linear flows
• Run calculations via HPC
• Passerelle workflows
• More, faster, better processing steps
![Page 30: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/30.jpg)
Try DAWN for yourself!
• DAWN 1.7.1, available now!
• http://www.dawnsci.org
• See directions on how to use DAWNSci – “Getting started with DAWNSci”
• Videos on how to do use slicing, data reduction, Python scripting, processing at https://www.youtube.com/user/DAWNScience
• Follow us on Twitter - https://twitter.com/dawnscience
• Read the article about us in the Eclipse newsletter - https://www.eclipse.org/community/eclipse_newsletter/2015/january/article3.php
• DAWN paper accepted for publication, doi: 10.1107/S1600577515002283
• Email us: [email protected]
![Page 31: The DAWNSci Eclipse science usecases Jun Aishima Software Engineer Diamond Light Source Didcot, UK](https://reader035.vdocument.in/reader035/viewer/2022070415/56649cab5503460f9496cd98/html5/thumbnails/31.jpg)
+1 0 -1
Sign in: www.eclipsecon.org
Evaluate the sessions