a collaborative approach to component-based community models and tools george leavesley and olaf...
TRANSCRIPT
A Collaborative Approach A Collaborative Approach to Component-Based to Component-Based
Community Models and Community Models and ToolsTools
George Leavesley and Olaf George Leavesley and Olaf David Colorado State David Colorado State
University, USDA, Fort Collins, University, USDA, Fort Collins, COCO
2
Overview
• Definitions
• Technical and Social (Cultural, Institutional, ..) Issues
• The Object Modeling System (OMS)– Component development
– Model assembly
– Collaborative Component and Model Development (Colab)
– Component and model sharing (the Repository)
– Support components (visualization, calibration, sensitivity analysis, …)
– Selected applications
• Closing Remarks
3
Definitions
• Component-based modeling - based on the concept of
composing or assembling models out of independent but
pluggable parts– Model components support -
• REUSE: can be reused in other models
• CONSISTENCY: provide consistent results
• QUALITY: can be rigorously tested
• FLEXIBILITY: appropriate code for problem objectives, data
constraints, and space-time scales of application
• MULTI-DISCIPLINARY APPROACH: experts focus on their
discipline and interfaces among disciplines
• Community – Hydrologic/environmental science and all
related disciplines? Research and operational users?
4
Technical Issues
• Multiple system architectures (portability)
• Numerous modeling frameworks
• Multiple programming languages
• Legacy models and legacy code (wrap, compile, refactor …)
• Open source
• Continuing advances in software and hardware
• Need highly experienced and skilled computer scientists as
a part of the community
• …
5
Social Issues
• Acceptance and awareness of the advantages of community modeling frameworks
• Agreement on appropriate and desirable framework technologies
• Concurrence about terminology across disciplines
• Organizational scope
• Team skills
• Framework/language learning curve
• Rewarding reuse, benefit it will have others
• Crediting papers vs. code
• Technology diffusion – how innovations are accepted by successive subgroups
6
Object Modeling System
• Component based IDE
for
– Developing model
components
– Integrating
components into
models
– Running models
– analyzing model
results
– Managing modeling
resources
7
Object Modeling System (OMS)Modeling Projects
Component Library
Assembled Model
Parameter Editor
ComponentEditor Output
Analysis
IDE based on NetBeans
8
Generic Modeling System Schematic
Data IO
GUI
Time stepcomponent
Spatial unitcomponent
DataParameterHandling
time step iteration
spatial unit iteration
ETP
Inter-ception
Snow
Soil-water
Ground-water
Inter-flow
Base-flow
SurfaceRO
Irrigation
Erosion
Surfacewater use
Groundwater use
Plantgrowth
Stre
am
RO
Generic SystemComponents
ModelSetup
SensitivityAnalysis
Optimization
Process modulelibrary
ETP
Hydr.
GW
…
…
…
WQWQ
Irrig.
…
…
[Krause 2004]
Component Development
“Creating the model building
blocks”
10
What is a Component ?
• Encapsulates operations to realize one specific purpose
• Typically a ‘physical’ process.
• Component attributes provide the communication among
components
Model
ComponentComponent
Attribute
Snow
Snowmelt
Soil
11
Component Implementation
• Java Class that implements
– Executable,
– Stateful …
• Defines Attribute references
– Attribute.Double
– Attribute.Integer
– ..
• Defined Method structure
– initialization
– component logic
– cleaning up resources
– additional methods
init()
run()
cleanup()
Component Implementation
Attribute objects
12
Component API
13
Component Meta Data
• Component Name
• Annotations
– Description
– Keywords
– Version Info
– Source
– Author
– (Status)
14
Component Attributes
• Attribute Objects
– Type
– Name
• Attribute Annotations
– Description
– Access
– Role
– Unit
– Constraint
• Packages– org.oms.model.data
– org.oms.model.data.annotations
15
Native Components
• Can be implemented in FORTRAN,C, C++
• FORTRAN: automatic encapsulation of
subroutines/functions into a component.
• Embedded JNative library is able to call dynamic link
libraries.
Model Building
- Integrate Components into a
model
- Connect Component Attributes
17
Adding a Component
• Drag and
Drop a
component
into a model
• Where to
instantiate
the
Attributes?
• Scoping
18
Create a Model
• A model is a
hierarchical
assembly of
components
• Sequential
Execution order
• Model Wizard
19
Component Attribute Connectivity• Introduced by components
when they are added to the
model or explicitly declared
• Attributes get shared among
components
• Global data element in a model
• Primitive and aggregated data
structures
• Connections changeable
• Component Attribute
references point to Model
Attribute Objects
20
Component and Attribute Connectivity
21
Component Palette
• Controls execution of
components
• Control Components
– Sequential execution
– Parallel Execution
– Conditional Exec
– Iteration
– Temporal Execution
– Spatial Execution
• Repository Components
22
Time – Space – Process Configurations
Space
Time
ProcessTime
Process
SpaceTime
Space
Process
23
Time – Space – Process Configurations
TimeSpace
Space
Processes
Processes
24
Modeling Projects
• Workspace that contains all resources for developing and running a model– Documentation– Input Data– Output data– Model Components
• Component = behavior– Model Assemblies
• Model = components + attributes– Simulations
• Simulation = model + input data + execution scheme + selected output analyses
• Project level Actions
25
Collaborative Model SupportCollaborative Model Support
• 1300+ users
• 20+ orgs
• 350+ projects
• since 2005
• USDA/USGS/
EPA/ARS/CSU/..
26
Modeling Project Support in CoLab
• Issue Tracking for
Bugs & Tasks
• Document
Management
• Wiki Spaces
• Version Control
(Subversion)
• QA metrics
• Build Support
• Discussion Forum,
Email Integration
• Search and Indexing
27
USDA Collaborative Modeling Infrastructure
Model DevelopmentModel Development- Construct- Run- Analyze- Test- Verify
Model ProjectModel Project ManagementManagement - Tracker- Forums- Documents- CMM Reports- Access Control
ModelModel ResourcesResources Change ManagementChange Management- Version Control- Change Management- Repository- Concurrent Access
OMSOMSOMSOMS
SubversionSubversionSubversionSubversion
CodebeamerCodebeamerCodebeamerCodebeamer
Component Repository
“Towards the establishment of a standard library of interoperable, reusable science
and auxiliary modeling components”
OMS Component Repository
OMS Model
Component Repository
Library of reusable components each described by:- Set of attributes
- Meta data on key details to enable retrieval
Represent / Publish (push)
Search/Retrieve (pull)
30
Access CoLab Components directly from OMS
Connect
ExplorePalette
Use
31
Model Support and Analysis Components
32
Map OMS model input/output data into NASA World Wind
33
Interactive Model Interface Tools
ProjectBrowser
SimulationNavigator
Data Table
EquationBox
MapWindow
HRU 15Selected
Data Table Filters
TimeSlider
34
Analysis Tools
• Canned analysis configurations
• Analysis of output data
– Graphs
• TimeSeries Plot
• XY Error
• Flow duration
• Residuals
– Statistics
• Table IO/API
35
ESP Trace Analysis
• Ensemble
Streamflow
Prediction (ESP) Tool
• Statistical analysis
• Report generation
36
Hay and Umemoto, 2006, Multiple-Objective Step-Wise Calibration using Luca: U.S. Geological Survey OFR 2006-1323.
Multiple-Objective Step-Wise Calibration Tool Using SCE
LUCA -
37
Selected Current Applications
38
Initially:Initially: Water Quality Water Quality Water ConservationWater Conservation Soil QualitySoil Quality
Later:Later: Air QualityAir Quality Wildlife HabitatWildlife Habitat
Initially:Initially: Cropland (Rainfed)Cropland (Rainfed)
Later:Later: Cropland (Irrigated)Cropland (Irrigated) GrazinglandsGrazinglands Wetlands Wetlands Agro-forestry LandsAgro-forestry Lands
Resource Concerns Land Use CategoriesResource Concerns Land Use Categories
USDA Conservation Effects Assessment Project (CEAP)
A Cooperative Effort to Assess Environmental Effects and A Cooperative Effort to Assess Environmental Effects and Benefits from Conservation Programs at National and Benefits from Conservation Programs at National and
Watershed ScalesWatershed Scales
39
SoilClimate
Plant/Crop …
NRCS Data Warehouse
OMS Model Interface
PS1
Nutrients
Hydrology 1
Crop Growth 1
PS2PS1
Erosion WindLeaching
1
Hydrology 2
Crop Growth 2
Hydrology 2
Crop Growth 1
OMS Model A OMS Model B OMS Model C
ARS Science Module WarehouseOMS Model Engine
OMS Toolbox
Hydrology
ErosionCrop
Growth
Modules..
ARS/CEAP Goal regionalized models, databases, and scenarios for site-specific watershed
assessment studies and national assessments
40
Agricultural Water Supply Forecasting
(USDA – NRCS National Water and Climate Center)
41
Moving from a regression-based seasonal supply
forecast to a process-model based daily/weekly/monthly
forecast approach using OMS
Agricultural Water Supply Forecasting
Research Elements:
•Evaluate alternative climate scenarios in ESP
•Evaluate alternative precip distribution methods
•Develop an ensemble of models for forecasting
42
Next Steps
• Continue to add components to the shared library from
research, support, and applications users
• OMS 3.0 will enable scalability from 1CPU to multi-core to
cluster to cloud computing
• OMS runtime will be part of USDA SOA Business Enterprise
architecture
43
Closing Remarks
• OMS is one of several frameworks focused on “community” model development
• Regarding technical issues OMS –– Supports multiple platform applications, open source
– Supports multiple programming languages using industry standard language support tools (minimizing use of design specific languages, minimizing learning curve)
– Supports scalability across single to multiprocessor computing approaches
• Is there a universal framework concept acceptable to the “community” or a need to consider multiple framework concepts?
• Support from the Computer Science community is critical in community model framework development
44
Closing Remarks
• Regarding Social Issues
– Concensus on the CUAHSI community vision needed
– Workshops are a great start
– Need to address Technology Diffusion in the process of Community
Model standards adoption (See Dibner and Arctur, IEMSS 2008)
• Innovators, Early Adopters, Early Majority, Late Majority, Laggards
• Theory says chasm is between Early Adopters and Early Majority
– Include all the anticipated community disciplines in the design
workshops
– While research is a focus, don’t exclude the applications community