quantifying thefuture
Post on 13-Jun-2015
137 Views
Preview:
TRANSCRIPT
Gio Wiederhold SimQL 1
Improving Decision-Making Support
by Linking Database results to Simulations
Gio Wiederhold Stanford University
July 2011
Gio Wiederhold SimQL 2
Problem : Mismatch
Database Technology should support Decision-Making
• What does database technology do?
o Databases provide information about past events » Consistent
» Reliable
» Fast
• What does a decision-maker do?
o Guess how decisions will affect the future
» Multiple possibilities
» Uncertainty
» Slow, manual, multiple tools
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 3
Information Systems should also Project into the Future
Support of decision-making requires dealing with the future , as well the past
• Databases deal well with the past
• Sensors can provide current status
• Spreadsheets, simulations deal with the likely futures
Information systems should be able to combine all three
time past now future
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 4
Decision-making (DM)
Analyze Alternatives
• Current Capabilities
• Future Expectations
• Planning for them
Process tasks:
• List resources
• Enumerate alternatives
• Prune alternative
• Compare alternatives
future now
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 5
• Data collection
• Data validation
• Data integration
• Information selection
• Data reduction & summarization
• File generation for analysts
Current Processes
• Data conversion to files for spreadsheets.
• Model building and testing by analysts
• Planning for likely future scenarios
• Recording expected results .
• Comparing many scenarios .
• Finding the best plans .
• Advising the actual . decision maker
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 6
Progress in Data Integration
Information Integration has progressed in supporting
Decision Making
1. Integrate data from distributed sources
o Issues: inconsistency of scope and timing
2. Capture new relationships o Often requires expert inter-domain knowledge
3. Include current sensor data o Select streaming data
4. include predictions about future courses
******* A new, potentially major topic *******
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 7
DM support is disjoint
Distribution
Planning Science
extensions to move
to networked support
are also disjoint
does not interoperate
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 8
Current state of DM Support
past now future
Data integration
distributed, heterogeneous
Databases
Intuition +
organized support disjointed support
• Spreadsheets
• Resource allocations
• Explicit simulations
various point assessments
x17 @qbfera
ffga 67 .78 jjkl,a
nsnd nn 23.5a
8/17/2012 Gio: SimQL
Past future time
Gio Wiederhold SimQL 9
Prediction Requires Tools
E-mail this book,
Alfred Knopf, 1997
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 10
Requirements for DM
• Ubiquitous access to simulations
of a wide variety of types
• Rapid response to parameter changes
oAccess to up-to-date facts
oMay need High-Performance recomputation
• Model, scenario, and choice retention
o Analysts’ planning to be reused
» But updatable
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 11
• Databases o High-level languages
» Data descriptions
o Drive detailed processes
o Intentional
• Simulations & spreadsheets o High-level languages
» Model desriptions
oParameter driven
o Extensional
8/17/2012 Gio: SimQL
How to merge 2 disciplines
Gio Wiederhold SimQL 12
• Enable intentional simulation access
o Follow database model » Similar to data description
o Provide interfaces »To support needed processes
Create SimQL similar to SQL
schema & links to access procedures
8/17/2012 Gio: SimQL
Integration concept
Gio Wiederhold SimQL 13
Transform Data to Information
value-added
services
decision-makers
8/17/2012 Gio: SimQL
Database Design
Data
Collection Model
Design Data-driven
Modeling Plans
Reports
o o
:-(
-)
oo
:-)
middle-
management Schema
SQL user
Gio Wiederhold SimQL 14
Stanford Experiment uses an existing SQL parser:
1. Replace the SELECT verb with ESTIMATE;
2. Remove the UPDATE statement. Nothing persists
3. Replace CREATE DATABASE with CREATE MODEL;
4. Add to the CREATE attributes IN, OUT, and INOUT;
5. Add a REGISTER statement to identify resources;
6. Replaced SQL’s functions code generators that access stored data with functions that deliver the
a. Query IN parameters to various simulations
b. Collect the data specified as OUT parameters
c. Return the result.
8/17/2012 Gio: SimQL
Language implementation
Gio Wiederhold SimQL 15
SQL:
SELECT Temperature, Cloudcover, Windspeed, Winddirection FROM
WeatherDB WHERE Date = `yesterday' AND Location = `ORD'.
SimQL:
ESTIMATE Temperature, Cloudcover, Windspeed, Winddirection FROM
WeatherSimulation WHERE Date = `tomorrow' AND Location = `ORD'.
8/17/2012 Gio: SimQL
Examples
Gio Wiederhold SimQL 16
Available Functions
1. Continously executing: weather prediction o SimQL result reports best match samples
2. Execution specific to query: Spreadsheet what-if assessment o may require HPC power for adequate response
3. Past simulations collect results in a base: materials o performs inter- or extra-polations to match query parameters
4. Combinations, i.e., 2. + 3.: top layer simulation using stored
partial lower level results: weapon performance in new setting
5. Human-in-the-loop: Wrapper for Amazon’s Mechanical Turk
Note
• A simulation service program can be written in any language
• A simulation service must be compliant to the interface spec.
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 17 8/17/2012 Gio: SimQL
System Concept Layout
Gio Wiederhold SimQL 18
Interfaces enable integration: SimQL to access Simulations
time past now future
Msg
systems,
sensors
Databases,
accessed via SQL or
XML, CORBA compliant
wrappers
Simulations,
accessed via SimQL and
compliant wrappers
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 19
Current State of SimQL research
Spreadsheets Weather Engineering
wrapper wrapper wrapper
Test Application
GUI collect language
requirements
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 20
Stanford Experiment Models
Spreadsheets Engineering
wrapper wrapper wrapper
Logistics
Application Manufacturing
Application
Weather
(short-, long-term)
wrapper
Test
Data
SimQL access SimQL access
SimQL access
SQL access
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 21
• Stanford experiment only produced point results.
• A decision maker would estimate multiple scenarios
1. Collect results identified with parameters
2. Provide search functions to compare results
1. Consider time lines for result synchronization
3. Support pruning of low-value results
4. Deliver only high-value results to decision-maker
8/17/2012 Gio: SimQL
More to be done
Gio Wiederhold SimQL 22
Use of Simulation Results
Simulation results can be composed for
alternative Courses-of-actions
Composition should include computation and recomputation of likelihoods
Likelihoods change as now moves forwards and eliminates earlier alternatives.
time 0.4
0.6
0.2
0.5
0.3
0.5
0.2
0.1
0.1
0.1
0.03
0.07
0.1
0.5 0.3
0.2
prob
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 23
• p=30% chance of rain
• Flight p=91% likely to arrive with 15 min of ETA
• Interest rate p=50% same, p=25% 1% higher, … .
• Employee p=50% returns to work in a week, … .
• Project p=10% completed in time, …
• Spreadsheets can compute alternative values with such data provided by the model builder, not the SimQL user.
8/17/2012 Gio: SimQL
Estimates have probabilities
Gio Wiederhold SimQL 24
The branches can be labeled with probabilities,
then assessed using the outcome with values
past now future
Next period alternatives
0.4
0.6 and subsequent periods
0.2
0.1
0.5
0.3 0.2
0.1
0.1
0.13
0.3
0.2
0.07
0.4
0.3
0.1
1000
2000
5000
1000
0
-6000
-3000
Values
100
600
1100 500
200 200
-420 0
-820 -400
1200
66
134
-1220
1266
-1086
time
prob
value
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 25
Integrating data & planning support will make
our data reusable and much more valuable
past now future
Re-assess as time
marches forward !
A Pruned Bush
Databases, . . . Spreadsheets,
other simulations,
Msgs
sensors
1000
2000
5000
1000
0
100
600
1100 500
200 200
0
1200
66
time
1266 ?
? ?
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 26
Even the present needs SimQL
time past now future
last recorded observations
simple simulations
to extrapolate data
Is the delivery truck in X?
• Is the right stuff on the truck?
• Will the crew be at X?
• Will the forces be ready to accept delivery?
point-in-time for situational assessment
Not all data are current:
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 27
Use of Simulation Results
Simulation results can be composed for
Alternative Courses-of-actions
Composition should be seamless, elegant, with computation and recomputation of likelihoods
Results change as now moves forwards and eliminates earlier alternatives.
Gio Wiederhold SimQL 28
Summary
Databases • serve clients via SQL by
Sharing a Model (The Schema)
A query language over the model the SQL interface enables • independence of
application development DBMS technology development reuse of infrastructure
Today
• most new systems use a DBMS for data storage
even with less performance, inability to handle all problems, but enough of them well enough.
Simulations should • serve clients via SimQL by
Sharing a Model (research q.) A query language over the model
a SimQL interface enables • independence of
application development simulation technology develop’t reuse of infrastructure
Objective
• build information systems combining DBMS, Simulations
even with less performance, inability to handle all problems, but enough of them . . .
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 29
Further research questions
• How to move seamlessly from the past to the future?
• How can multiple futures be managed (indexed)?
• How can multiple futures be compared, selected?
• How should joint uncertainty be computed?
• How can the NOW point be moved automatically?
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 30
Future information systems
Combine data from the past, with current data, knowledge, and predictions into the future
o o o o
o o
Assessment of the
values of alternative
possible outcomes
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 31
SimQL research questions
• How little of the model needs to be exposed?
• How can defaults be set rationally?
• How should expected execution cost be reported?
• How should uncertainty be reported?
• Are there differences among application areas that require different language structures?
• Are there differences among application areas that require different language features?
• How will the language interface support effective partitioning and distribution?
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 32
Moving to a Service Paradigm
• Server is an independent contractor, defines service
• Client selects service, and specifies parameters
• Server’s success depends on value provided
• Some form of payment received for services
x,y
Databases are a current example.
Simulations have the same potential.
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 33
Summary of SimQL
A new service for Decision Making:
• follows database paradigm – ( by about 25 years )
• coherence in prediction – displacement of ad-hoc practices
• seamless information integration – single paradigm for decision makers
• simulation industry infrastructure – investment has a potential market
– should follows database industry model:
Interfaces promote new industries
8/17/2012 Gio: SimQL
Gio Wiederhold SimQL 34
Gio Wiederhold: "Information Systems that Really Support Decision-making"; 11th International Symposium on Methodologies for Intelligent Systems (ISMIS), Warsaw Poland, June 1999, in Ras & Skowron Foundations for Intelligent Systems, Springer LNAI 1609, pages 56-66
Gio Wiederhold and Rushan Jiang: “Augmenting Information Systems with Access to Predictive Tools”;
http://infolab.stanford.edu/pub/gio/2000/VLDB2000-1.htm
The specifics of the language as implemented are at
http://www-db.stanford.edu/LIC/SimQL.html
8/17/2012 Gio: SimQL
Publications
top related