the state of nexus
DESCRIPTION
The State of NeXus. P.F. Peterson and F.A. Akeroyd on behalf of the NeXus International Advisory Committee. 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008. NIAC Berkeley, CA. NIAC Villigen, Switzerland. NIAC Berlin, Germany. NIAC Sydney, Australia. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/1.jpg)
The State of NeXus
P.F. Peterson and F.A. Akeroydon behalf of the NeXus International
Advisory Committee
![Page 2: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/2.jpg)
NeSSI: Collaborative Software and Other Mythical Creatures
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
Soft
NeS
S’94
Soft
NeS
S’95
Soft
NeS
S’96
v1.1
.0, v
1.2.
0, v
1.2.
1, v
1.3.
0
v1.3
.1, v
1.3.
2, v
1.3.
3
v2.0
.0
v2.1
.0
v3.0
.0
v4.0
.0
NIA
C Pa
sade
na, C
A
NIA
C Vi
llige
n, S
witz
erla
nd
NIA
C Sa
nta
Fe, N
M
NIA
C G
reno
ble,
Fra
nce
NIA
C Be
rkel
ey, C
A
NIA
C Be
rlin,
Ger
man
y
NIA
C Sy
dney
, Aus
tral
ia
![Page 3: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/3.jpg)
NeSSI: Collaborative Software and Other Mythical Creatures
![Page 4: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/4.jpg)
NIAC Members• Frederick Akeroyd, Rutherford Appleton Laboratory, UK (ISIS, Executive Secretary) • Stephen Cottrell, Rutherford Appleton Laboratory, UK (Muon Representative) • Matthias Drochner, Forschungzentrum Jülich, Germany • Emmanuel Farhi, Institut Laue Langevin • Andrew Götz, European Synchrotron Radiation Facility, France • Nick Hauser, Australian Nuclear Science and Technology Organisation, Australia
(Documentation Release Manager, netCDF contact) • Jens-Uwe Hoffmann, Helmholtz Zentrum Berlin (renamed HMI), Germany • Pete Jemian, Advanced Photon Source, USA • Paul Kienzle, DANSE, USA • Mark Könnecke, Paul Scherrer Institut, Switzerland (Technical Committee Chair) • Nick Maliszewskyj - NIST Center for Neutron Research, USA • Raymond Osborn, Argonne National Laboratory, USA • Peter Peterson, Spallation Neutron Source, USA (Chair) • Stephane Poirier, Synchrotron Soleil, France • Thomas Proffen, Los Alamos National Laboratory, USA • Jiro Suzuki, KEK, Japan
![Page 5: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/5.jpg)
Principles of NeXus
Mission: NeXus enables comprehension of data amongst the neutron, x-ray, and muon science communities through a standard data format.
![Page 6: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/6.jpg)
Reduced need for local expertise
• [email protected]• [email protected]• http://www.nexusformat.org
• http://svn.nexusformat.org• http://trac.nexusformat.org
![Page 7: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/7.jpg)
Simple Plotting
NXdataNXdata
![Page 8: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/8.jpg)
Reduced need for conversion
![Page 9: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/9.jpg)
Is it based on anything?
• Hierarchical Data Format (4 and 5)– NXopen(filename,NXACC_CREATE4,handle)– NXopen(filename,NXACC_CREATE5,handle)
• eXtensible Markup Language– NXopen(filename,NXACC_CREATEX,handle)
– NXopen(filename,NXACC_RDWR,handle)– NXopen(filename,NXACC_READ,handle)
![Page 10: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/10.jpg)
Who is committed to NeXus
SNS at ORNLSNS at ORNLNCNR at NISTNCNR at NISTISIS at RAL (neutrons and muons)ISIS at RAL (neutrons and muons)DIAMOND at RALDIAMOND at RALBI at ANSTOBI at ANSTOFRM2 at TUMFRM2 at TUMKENS at KEKKENS at KEKJ-PARCJ-PARC
APS at ANLAPS at ANLHMI in BerlinHMI in BerlinMLNSC at LANLMLNSC at LANLSINQ at PSISINQ at PSIILL in GrenobleILL in GrenobleLLB at SaclayLLB at Saclay
![Page 11: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/11.jpg)
Existing HDF4/5 tools
• HDFLook – HDF based plotter (???)• Java HDF Viewer (NCSA)• Scientific Data Browser – view data over the
web (NCSA)
• Any web browser• Eclipse with WTP plugin
Existing XML tools
![Page 12: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/12.jpg)
NeXus Utilities• NXbrowse – CLI tool to interactively explore NeXus files• NXdir – CLI tool that lists contents of NeXus file and convert
selected portions to ASCII• NXextract• NXtoDTD – Creates a definition file from an existing NeXus file• NXtoXML – Writes an XML version of an existing NeXus file• NXtranslate – anything to NeXus converter NXvalid – GUI tool to
interactively explore, plot, and validate NeXus files• NXtree – Create a html based tree view of NeXus files• NXvalid – GUI tool to view and validate NeXus files• NXvalidate – command line tool for validating structure of
NeXus files
![Page 13: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/13.jpg)
NXvalid
![Page 14: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/14.jpg)
NXtranslate• Anything to NeXus converter
– Binary dump– FRM2– IPNS run– NeXus– Spec– XML
• Uses an XML-based translation file to determine where to get information from
• Additional information can be supplied in the translation file directly• CLI to encourage batch usage• Simple macro language to allow translating several files using one
translation file
![Page 15: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/15.jpg)
Tools and Utilities• addit, subi – programs to add or subtract powder data and write files suitable for Rietveld programs (SINQ)• Amortool – simple CLI tool for Reflectometer data analysis (SINQ)• anatric, cami4pcd – four circle diffractometer data analysis (SINQ)• BerSANS – data analysis for SANS through adapter (SINQ)• DANSE – Data Analysis Architecture for Neutron Scattering Experiments (CalTech)• DAVE (future) - Data Analysis and Visualization Environment (NCNR)• FABLE (ESRF)• fit – home grown fitting program (SINQ)• GumTree – scientific workbench for instrument control (BI)• hdfb.sav and h5b.sav – browser for any 1D/2D/3D data saved in HDF (APS)• ISAW – Integrated Spectral Analysis Workbench (IPNS)• KUPLOT – General purpose plotting program (MLNSC)• LAMP – Large Array Manipulation Program (ILL)• MANTID – Manipulation Toolkit for ISIS Data (ISIS)• McStas – instrument modeling program (SINQ/ILL)• Nathan – TOF data analysis for FOCUS (IDL based) (SINQ)• NDS – NeXus data server, publish NeXus files on TCP/IP, read only (SINQ)• ninx – inx (ILL) adapted to read FOCUS TOF files (SINQ)• NNDB – Java program to browse NeXus files published through NDS (SINQ)• NXviewer – OpenDave based viewer (FRM2)• OpenGenie – General purpose neutron and muon scattering data reduction and analysis tool (ISIS)• Redas – data analysis for Reflectometer, based on Scilab (SINQ) • ReflPak – data analysis for Reflectometer (future) (NCNR)• Scilab – general matlab like data analysis tool - with a NeXus interface for that (SINQ)• SNS Portal (SNS)• UDA and WIMDA – General purpose muon scattering analysis packages (ISIS)
![Page 16: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/16.jpg)
![Page 17: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/17.jpg)
nexusformat.org
![Page 18: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/18.jpg)
NeXus API v4.2
![Page 19: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/19.jpg)
NeXusDefinitionLanguage
![Page 20: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/20.jpg)
![Page 21: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/21.jpg)
![Page 22: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/22.jpg)
![Page 23: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/23.jpg)
![Page 24: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/24.jpg)
Validation of NeXus files using XMl Schema
Freddie AkeroydNIAC
![Page 25: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/25.jpg)
Why Validation?
Standardised definitions make sharing files and analysis programs easier
However, developers need a way to confirm their files comply to the standard
![Page 26: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/26.jpg)
NeXus Meta-DTD Format
Not really a true DTD, more a template XML file
However there are a few issues:Syntax is a little “cryptic” – required training
to readNot easily machine parsableValidated by humans
![Page 27: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/27.jpg)
Base classes /dictionaries
Base classes /dictionaries
Application definitionsApplication definitions
DocumentationDocumentationValidation formValidation form
schemaschema
![Page 28: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/28.jpg)
<definition name="NXfermi_chopper" type=“group” extends="NXchopper"> <field name="type" type="NX_CHAR">
<doc>fchopper type</doc> </field> <field name="radius" type="NX_LENGTH">
<doc>radius of chopper</doc> <attribute name=“colour” /></field> <group type="NXgeometry">
<doc>geometry of the fermi chopper</doc> </group> …
New Definition Format
![Page 29: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/29.jpg)
NXDL post XSLT transform
![Page 30: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/30.jpg)
NeXus Schema (XSD) File
![Page 31: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/31.jpg)
Advantages of XML schemaHave built-in data types and allow new types
Support extension and restriction of typesMany tools exist for using schema to validate
XML filesTools exist for generating language bindings
and data reading libraries from schemaPossible to add constraints on values via
enumerations, patterns and rangesSupports keys and uniqueness constraints
![Page 32: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/32.jpg)
How are definitions built? Each NX* base class is defined in a separate file
This separation will make life easier for future base class editors
Instrument definitions restrict NXentryType by marking required items as mandatory
Inheritance used to ensure consistency and avoid duplication: “object”, “component”, “chopper base”
Enumerations created for chopper types, units, shapes, etc.
Special “Types with units” defined, such as floatAngleDataType and floatLengthDataType
![Page 33: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/33.jpg)
XML fileXML file
NXD Definition
file
NXD Definition
file
NeXus fileNeXus file
XML schema
XML schema
Validation (xmllint)
Validation (xmllint)
NXVALIDATE
![Page 34: The State of NeXus](https://reader036.vdocument.in/reader036/viewer/2022062801/56814411550346895db0aebd/html5/thumbnails/34.jpg)
Questions?Questions?
NeXus homepagehttp://www.nexusformat.org
Feature requestshttp://trac.nexusformat.org