Download - ICAT Integration at ISIS
![Page 1: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/1.jpg)
ICAT Integration at ISIS
Tom Griffin, ISIS FacilityICAT Developer WorkshopThe Cosener’s House, Abingdon
August 2009
![Page 2: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/2.jpg)
ISIS
• Neutrons and Muons• Operating since 1984• 1600 user base; ~200 staff• Volume of data ~6TB• ~5M datafiles (27 instruments,
500/hour)• >6.7GB metadata, >33M rows
![Page 3: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/3.jpg)
Current Pre-Experiment
![Page 4: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/4.jpg)
Data.ISIS
![Page 5: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/5.jpg)
Data.ISIS
• Main point of access for ISIS data• Maintains index of Run number ->
Folder– GEM12345– \\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw– .txt .log .sav0
• Ranges• Zip files
![Page 6: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/6.jpg)
![Page 7: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/7.jpg)
Data.ISIS Authentication
Data Portal
ICAT API
Data.ISIS
User performs search via application e.g. Data Portal
Search is executed in ICAT
Permitted results are returned to application
11
11
22
33
33
22
Results are displayed to the user44
44
55
User performs request to download datafile, multiple datafiles or dataset
55
66
ICAT creates http GET link and passes to back to user (routed through application)
sessionIdemail (optional)fileId(s) or datasetIdaction (i.e. download, zip,
compressed)
66
User clicks http link77
77
Data.ISIS call ICAT API to check permissionssessionId & datafileId(s) or
datasetId
88
88
Return Exception on failure or DownloadObject on success- userId- array [filename, cycle, run
number]
99
99
User gets their data!1010
1010
![Page 8: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/8.jpg)
Current Pre-Experiment
![Page 9: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/9.jpg)
‘To Be’ Pre-Experiment
![Page 10: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/10.jpg)
Problems
• RB Numbers• Business System
• Not released to users• Replace DataPortal with TopCat
![Page 11: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/11.jpg)
Related Projects
• Grid MP – distributed PC network• Analysis and simulation• New portal required anyway ->
1) Single point of access for Grid MP2) Use ISIS data in Grid jobs3) Store results and process in ICAT
Where to store data?
![Page 12: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/12.jpg)
Freddie Akeroyd (Data.ISIS)Chris Moreton-Smith (ISIS DFS/FRS)Hannah Guest (Facilities Business System & ERAS)Matt Clarke (SECI modifications)
Damian FlanneryStuart Ansell (writeRaw)
Laurent Lerusse (NXIngest)
Database Services, Roger Downing, FBU IT Service Desk (servers and hosting)
Core ICAT – Glen, Shoaib, Michael.....
Acknowledgments(who did what at ISIS)
ISIS Computing Group
ISIS Data Analysis Group
![Page 13: ICAT Integration at ISIS](https://reader035.vdocument.in/reader035/viewer/2022081506/56813d3e550346895da6fd4d/html5/thumbnails/13.jpg)
Questions...