Download - A Plan for a Plan
![Page 1: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/1.jpg)
FSP: Validation 7/15/09 1
FSP: Validation Program Planning
Presented by Martin Greenwald
FSP Kick-off MeetingPrinceton, 7/15/2009
![Page 2: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/2.jpg)
FSP: Validation 7/15/09 2
A Plan for a Plan
Refine our concept of what needs to be done
Enumerate a concrete set of tasks and deliverables
– Who does which tasks?
– How will they be carried out?
– Estimate resource requirements (for FSP definition phase)
Define a preliminary schedule
![Page 3: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/3.jpg)
FSP: Validation 7/15/09 3
Work Elements for FSP Validation Program Definition
Build team
Gap analysis
– Outreach to fusion community
– Assess and document lessons learned from other communities
Define validation planning tasks
– Detail tasks and roles for FSP validation
– Define mechanisms for collaboration with experimental groups
– Coordinate with code groups
Plan and prototype documentation strategy
Plan and execute pilot projects as needed
![Page 4: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/4.jpg)
FSP: Validation 7/15/09 4
Team Building
Successful validation program for FSP will require unprecedented level of collaboration between theory, modeling, experiments and diagnostics
Approximately 1 FTE explicitly funded for project definition
Will need to leverage other resources and build extended team from both inside and outside FSP
– Define responsibilities for verification and validation within FSP
– Identify and recruit collaborators from outside
– Provide modes of communication, documentation
Web site, Wiki, etc.
![Page 5: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/5.jpg)
FSP: Validation 7/15/09 5
Outreach Within MFE Community
Questions:
– Assess state of art and plans for validation in MFE
– How does community view priorities?
– How would they approach collaboration with FSP?
– How to best marshal necessary resources (outside FSP)?
– How to coordinate? Avenues for information gathering
– BPO/TTF validation task force
– ECC
– Other topical groups?
– ITER modeling team
– International fusion programs
– Informal get-togethers at meetings
– Site visits Build formal and informal collaborations
![Page 6: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/6.jpg)
FSP: Validation 7/15/09 6
Assess Lessons Learned from Other Communities
Communities
– Climate
– Combustion
– ASCII
– Other CFD
– Others?
Approaches
– Literature search
– Contacts (especially team members with outside contacts)
– Site visits?
![Page 7: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/7.jpg)
FSP: Validation 7/15/09 7
Perform Gap Analysis
Combine info from inside and outside MFE
Identify critical gaps in capabilities or methodologies, for example
– Post-processing and Visualization tools
– Analysis and synthetic diagnostics
– Other software requirements
– Statistical techniques
– Metrics
– Experimental capabilities
– Measurement capabilities
Manpower and other resource estimates (for FSP itself)
![Page 8: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/8.jpg)
FSP: Validation 7/15/09 8
Detail Tasks and Roles for FSP Validation
Tasks would include
– Definition of critical physics tests, priorities
– Design of validation experiments including diagnostic/experimental requirements
– Code predictions
– Experiment execution
– Analysis/tool development
– Documentation
– Feedback into code development effort
Define roles for theory, code groups, analysts, experimentalists, diagnosticians
Software support
Coordination and management
![Page 9: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/9.jpg)
FSP: Validation 7/15/09 9
Define Methodologies for FSP Validation
Summarize best practices
Elements include:
– Roles and responsibilities
– Jointly designed experiments – principles and practice
– Hierarchy of experimental tests
– Documentation of code predictions
– Measurement and documentation requirements for experiments
– Quantification of errors and uncertainties
– Quantification of comparisons
– Documentation methodologies
![Page 10: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/10.jpg)
FSP: Validation 7/15/09 10
Critical Need: Analysts
In “high-consequence” applications, V&V is led by dedicated analysts
– Weather prediction and climate modeling
– Turbomachinery
– Airframe design
Not tied to code development groups
Can serve as honest brokers, providing unbiased and dispassionate assessments
Requires specialized skills
Analysts would have leading role in designing and analyzing validation experiments
– Close collaboration with theorists, computationalists and experimentalists
– Help marshal the computational and experimental resources
– Help develop post-processors, synthetic diagnostics
What is FSP role in developing/training?
![Page 11: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/11.jpg)
FSP: Validation 7/15/09 11
Define Mechanisms for Collaboration with Experimental Groups
How to marshal necessary resources and build engagement and commitment
How to impact long term planning and development of capabilities
– Machine operations (heating, fueling, current drive, etc)
– Diagnostic development and deployment
How to organize experimental programs and interact with experimental planning
How to jointly design and execute validation experiments, including careful documentation
Identify experimental time, manpower and other resources required
Develop guidelines for publication and other IP issues
Define approach for coordination and management
![Page 12: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/12.jpg)
FSP: Validation 7/15/09 12
Coordination with Code Groups (Inside and Outside FSP)
Identify critical physics for testing
– Identify validation requirements at various “levels” of physics integration
Define validation experiments and measurements required
Coordinate with verification efforts
– make sure this has sufficient priority and is carried out in timely manner
“Regularize” interactions
– Validation results should guide code development
– Ensure adequate computational resources for verification and validation
Common infrastructure
– Documentation – approach, schema
– Data structures, API
– Post-processors, analysis tools, visualization tools and synthetic diagnostics
![Page 13: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/13.jpg)
FSP: Validation 7/15/09 13
Embody V & V Planning Within FSP
As FSP plan solidifies, map out research needs and directions in detail
Ask some important questions
– How will model predictions be used?
– Which applications?
– What are the impacts of predictions? of errors in predictions?
Assess Status
– Which areas are well understood? Where are they uncertain or controversial?
– What new developments in physics or methodology are required?
Define requirements for validation
– When are models “ready”?
– Which are the critical elements of each model? – priorities for testing
– What are the experimental and diagnostic requirements?
![Page 14: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/14.jpg)
FSP: Validation 7/15/09 14
Plan and Prototype Documentation Strategy (1)
What needs to be saved?
– (Everything)
– Include both experimental and modeling data
Raw and processed data
All auxiliary data and inputs, calibrations, assumptions, geometry, boundary and initial conditions, etc.
Estimations of errors and uncertainties
– Include metadata for every data item. Create a complete, coherent, self-descriptive structure
– Results of all analysis
– Textual information to describe methods, physics, commentary.
– Everything time stamped and attributed.
Make this list complete, specific then prototype
![Page 15: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/15.jpg)
FSP: Validation 7/15/09 15
Plan and Prototype Documentation Strategy (2)
Characteristics for data storage and access
– Needs to be shareable, easy to use (API critical), archival
– Dynamic and interactive – able to be updated, annotated, appended
– Queryable - searchable by content or by address
– Browsable
– Linked to publications
– ?
Refine functional and non-functional requirements, define approach and prototype
(With other groups) define data access rules
![Page 16: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/16.jpg)
FSP: Validation 7/15/09 16
Plan and Execute Pilot Projects
How could pilot projects aid FSP? (more or less in order of importance)
– Help address open issues or questions
– Validate approaches and methodology
– Start to build needed collaborations
– Demonstrate ROI of FSP
Pilot projects chosen to meet above criteria
Workshops?
Encourage and solicit proposals?
![Page 17: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/17.jpg)
FSP: Validation 7/15/09 17
Some Short-Term Actions
Build team and define roles
Provide tools for collaboration and documentation (Should be FSP-wide, but structured?)
Enumerate deliverables
Define coordination with verification efforts
Define preliminary schedule
![Page 18: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/18.jpg)
FSP: Validation 7/15/09 18
End
![Page 19: A Plan for a Plan](https://reader036.vdocument.in/reader036/viewer/2022082817/56812bd4550346895d903729/html5/thumbnails/19.jpg)
FSP: Validation 7/15/09 19
Elements of Thrust: Relation of Processes
Schlesinger 1979
Imperfectly and
Incompletely Measured!