why do models fail? problems, problems …
DESCRIPTION
Why do models fail? Problems, problems …. John Rees BGS Head of Policy and Science Co-ordination Andrew Hughes BGS Groundwater Modeller and NMPI Co-organiser. Ideal vs reality. Generic issues as seen from a government institute - the British Geological Survey. - PowerPoint PPT PresentationTRANSCRIPT
Kingsley Dunham CentreKeyworthNottingham NG12 5GGTel 0115 936 3100
© NERC All rights reserved
Why do models fail?Problems, problems …
John ReesBGS Head of Policy and Science Co-ordination
Andrew HughesBGS Groundwater Modeller
and NMPI Co-organiser
© NERC All rights reserved
Ideal vs reality
© NERC All rights reserved
Generic issues as seen from a government
institute - the British Geological Survey
© NERC All rights reserved
BGS and numerical modelling
Groundwater - Model Developer and user Coastal change - Poacher and gamekeeperContamination - often the ‘honest broker’
© NERC All rights reserved
Typical reasons for failure
• Lack of consistency of approach
• Turnover and education of staff in client organisations
• Language between user and modellers, and between modellers
• Appropriate and sufficient data
• Trust between parties
• Unrealistic expectations
• Honesty about the limitations of models
• Cost of taking-up new developments
© NERC All rights reserved
\Policy makers
\Resource Managers
\Modellers
© NERC All rights reserved
Key organisations
• Universities
• Consultants
• Government Institutes
• Regulators
• Utilities
• Government Ministries
• Pan-governmental
© NERC All rights reserved
Regulator issues
• Limited number of staff that have modelling skills
• Accessibility of models is often poor – they are often not easy to run
• The maintenance and updating of models has a high cost (the leaky roof syndrome)
• Inadequate data to support development or understanding
• Expectations are often not met
• Stakeholders are not adequately consulted
© NERC All rights reserved
Resource manager issues
• Common uncertainty about the specification and scale of model required
• Lack of clarity about the needs of regulators
• Find that models have too much uncertainty for detailed use
• Inadequate timely stakeholder involvement
• Find that models are not as flexible as managers would like
© NERC All rights reserved
Model developer issues
• Models are driven by policy and not the other way round e.g. Habitats Directive
• Few examples of models being trusted by all parties
• Stakeholders are recognised as important, but involvement is very variable
• Personalities are important in defining how much stakeholders interact
• Managing expectations is very important
• Those consultants who deliver to spec and on time may not be the best to drive modelling forwards
© NERC All rights reserved
Broader implications
• Lack of efficiency associated with development encourages adoption or tweaking of older ‘industry standard’ models instead of adoption of newer models.
• Drive to conservatism encourages usage of ‘tried-and-tested’ consultants who focus on delivery, rather than more innovative scientists who will introduce new concepts and stretching the modelling.
• Acceptance of limitations (e.g. empirical constants, black-boxes) instead of driving better modelling.
© NERC All rights reserved
Towards solutions• Need common understanding before attempting to
formulate solutions
- Climate change models are not taken up by some national governments that do not accept reality!
• Language – differs markedly between disciplines
• Definition of problem type, organisations involved and geographic extent
• Problem is potentially huge, so need boundaries
• Guidelines are needed
© NERC All rights reserved
Summary
© NERC All rights reserved
Questions ?