veronica nieves civil and environmental engineering university of california, irvine jpl-nasa,...
TRANSCRIPT
Veronica NievesCivil and Environmental Engineering
University of California, Irvine
JPL-NASA, Pasadena, March 2nd, 2011
Maximum Entropy Principle Reveals Maximum Entropy Principle Reveals
Simplicity Behind ComplexitySimplicity Behind Complexity
Maximum Entropy Principle Reveals Maximum Entropy Principle Reveals
Simplicity Behind ComplexitySimplicity Behind Complexity
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt OverviewMaxEnt Overview
MAXIMUM ENTROPYMAXIMUM ENTROPY(MaxEnt) PRINCIPLE(MaxEnt) PRINCIPLE
APPLICATIONSAPPLICATIONS
FUTURE RESEARCHFUTURE RESEARCH
SUMMARYSUMMARY
APPLICATIONSAPPLICATIONS
FUTURE RESEARCHFUTURE RESEARCH
SUMMARYSUMMARY
2
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Identification of Identification of
ESSENTIAL PHYSICS.ESSENTIAL PHYSICS.
MaxEnt MaxEnt PrincipleMaxEnt MaxEnt Principle
GUESS -
PHYSICAL ASSUMPTIONS
Testable information given by
experimental results or conserved quantities.
ENTROPY -
MISSING INFORMATION
Measure of average amount of missing
information (or uncertainty) of random variable.
STATISTICS STATISTICS of random variable.of random variable.
MaxEnt: the probability distribution best representing the current state
of knowledge is the one associated to the largest entropy - subject to
physical constraints.
(Jaynes, 2003; Gregory, 2005).
(Dewar, 2009).
3
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt MaxEnt PrincipleMaxEnt MaxEnt Principle
MaxEnt allows to identify the essential information, isolate it from the rest
and still describe the whole system.
Why is MaxEnt important?
MaxEnt is useful when complete information is not available or when the
computational efficiency is important.
4
GUESS - PHYSICAL ASSUMPTIONS:
ENTROPY - MISSING INFORMATION:
p(x)
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt MaxEnt FormulationMaxEnt MaxEnt Formulation
* Lagrangian Multipliers *
5
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Example: closed system in thermal equilibriumMaxEnt Example: closed system in thermal equilibrium
p(x)
ENTROPY - MISSING INFORMATION
Gibbs’ canonical distribution:
GUESS - PHYSICAL ASSUMPTIONS:
(Tribus, 1961).
6
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEntMaxEnt
APPLICATIONSAPPLICATIONS
FUTURE RESEARCHFUTURE RESEARCH
SUMMARYSUMMARY
MAXIMUM ENTROPYMAXIMUM ENTROPY(MaxEnt) PRINCIPLE(MaxEnt) PRINCIPLE
7
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Application: drainage areaMaxEnt Application: drainage area
* Kapur, 1989:
“When the geometric mean is prescribed, the MaxEnt probability distribution is the power-function distribution.”
* Tarboton et al., 1989:
“Contributing area shows a power law relationship”.
Drainage area: the geographical area drained by a river and its tributaries.
p(x)
ENTROPY - MISSING INFORMATION
GUESS - PHYSICAL ASSUMPTIONS:
Geometrical mean
8
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Empirical vs. MaxEnt scaling exponentsMaxEnt Empirical vs. MaxEnt scaling exponents
=
9
River networks in Puerto Rico derived from USGS DEM data.
“Predicted” refers to (log(Ag/A1))-1, and “fitted” to their curve-fitting values.
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Empirical vs. MaxEnt scaling exponentsMaxEnt Empirical vs. MaxEnt scaling exponents
10
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Application: scale-invariant processesMaxEnt Application: scale-invariant processes
Similar scaling relation found for different separation distances on a large enough landscape (Rodriguez-Iturbe, 1997; Peters-Lidard et al., 2001).
Z1
Z2
r
Z1
Z2
r
Z1
Z2 r
GUESS - PHYSICAL ASSUMPTIONS:
Z1
Z2
r
ENTROPY - MISSING INFORMATION
p(x)
Geometrical mean
Multiscaling moments
11
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt DataMaxEnt Data
AMSR-E SSM for October 18th, 2009.
- Input brightness temp. at 10.7 GHz.
- Soil moisture in the top ~1 cm
(vertical sampling depth and averaged over 60 km horizontal spatial extent).
- Accuracy ~ 0.06 g/cm3.
- Typical day is of 28 half-orbits coverage.
- Swath width is 1445 km.
- 25 km EASE-grid cell spacing.
NED Shaded Relief Map (1 arc-second).
- Raster elevation data by USGS.
- Updated on a two month cycle and derived from diverse sources.
- Elevation values in meters referenced to NAD83 (horizontal
datum) and NAVD88 (vertical datum).
- Vertical accuracy ~ 7-15 m (depending on source DEM).
- Resolution of ~ 30 m (1 arc-second).
* http://nsidc.org/data/amsre
* http://ned.usgs.gov
12
Empirical
Theoretical q=2
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Empirical vs. MaxEnt distributionsMaxEnt Empirical vs. MaxEnt distributions
Nieves et al., Phys. Rev. Lett., 105 (2010). This paper was highlighted in the CEE UCI news: http://www.eng.uci.edu/news/2010/11/cee-paper-published-physical-review-letters 13
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Empirical vs. MaxEnt distributionsMaxEnt Empirical vs. MaxEnt distributions
Empirical
Theoretical q=2
Nieves et al., Phys. Rev. Lett., 105 (2010). This paper was highlighted in the CEE UCI news: http://www.eng.uci.edu/news/2010/11/cee-paper-published-physical-review-letters 14
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEntMaxEnt
APPLICATIONSAPPLICATIONS
FUTURE RESEARCHFUTURE RESEARCH
SUMMARYSUMMARY
MAXIMUM ENTROPYMAXIMUM ENTROPY(MaxEnt) PRINCIPLE(MaxEnt) PRINCIPLE
20
- MaxEnt is an inference algorithm that reveals essential information of complex
systems such as river networks, soil moisture, and topography.
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt SummaryMaxEnt Summary
Observable parameters.
Characterization of river basin properties.
Testable information: experimental results or conserved quantities.
E.T Jaynes, 1982.
- Microscopic physics can be avoided and the computational efficiency improved.
* Description of the whole system using a reduced data set that carry enough physical information to “mimic” nature behavior.
21
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEntMaxEnt
APPLICATIONSAPPLICATIONS
FUTURE RESEARCHFUTURE RESEARCH
SUMMARYSUMMARY
MAXIMUM ENTROPYMAXIMUM ENTROPY(MaxEnt) PRINCIPLE(MaxEnt) PRINCIPLE
22
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Future ResearchMaxEnt Future Research
* Acknowledgements: Acknowledgements: This work was supported by the postdoctoral research Balsells - Generalitat de
Catalunya Fellowship and the U.S. Army RDECOM ARL Army Research Office under Grants No. W911NF-07-
1-0126 and W911NF-10-1-0236.
- Design of a MaxEnt-based monitoring networkDesign of a MaxEnt-based monitoring network:: MaxEnt helps to select optimal observation sites by
providing a measure of information gaininformation gain (Nieves et al.,
in preparation).
- Generation of heat fluxes over landmasses, Generation of heat fluxes over landmasses,
oceans and snow/ice caps using the oceans and snow/ice caps using the MaxEnt MaxEnt
Production (MEP)Production (MEP): : only three variables (radiation,
temperature, and humidity) are needed (J. Wang et al.
2009 and 2011).
23
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt Future ResearchMaxEnt Future Research
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
Ma
xim
um
E
ntr
opy
M
axi
mu
m
En
tro
py
MaxEnt ReferencesMaxEnt References
E.T. Jaynes, Probability Theory: The Logic of Science, Cambridge Univ. Press, 2003.
P.C. Gregory, Bayesian logical data analysis for the physical sciences, Cambridge Univ. Press, 2005.
J.N. Kapur, Maximum entropy models in science and engineering, John Wiley & Sons, New York, 1989.
V. Nieves, J. Wang, R. Bras, and E. Wood, Maximum Entropy Distributions of Scale-Invariant Processes, Phys. Rev. Lett., 105 (2010), pp. 118701.
B.B. Mandelbrot, The fractal geometry of nature, W H Freeman and Co., New York, 1983.
D. Veneziano and J.D. Niemann, Self-similarity and multifractality of fluvial erosion topography 1. Mathematical conditions and physical origin, Water Resour. Res., 36 (2000), pp. 1923.
D. Veneziano and J.D. Niemann, Self-similarity and multifractality of fluvial erosion topography 2. Scaling properties, Water Resour. Res., 36 (2000), pp. 1937.
R.C. Dewar, Maximum Entropy Production as an Inference Algorithm that Translates Physical Assumptions into Macroscopic Predictions: Don't Shoot the Messenger, Entropy, 11 (2009), pp. 931.
S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, 1999.
D.G. Tarboton, R.L. Bras, and I. Rodriguez-Iturbe, Scaling and elevation in river networks, Water Resour. Res., 25 (1989), pp. 2037.
V. Nieves and A. Turiel, Analysis of ocean turbulence using adaptive CVE on altimetry maps, J. Marine Syst., 77 (2009), pp. 482.
B. Lashermes, E. Foufoula-Georgiou, and W. Dietrich, Channel network extraction from high resolution topography using wavelets, Geophys. Res. Lett., 34 (2007), pp. L23S04.
A. Turiel and N. Parga, Multifractal wavelet fillter of natural images, Phys. Rev. Lett., 85 (2000), pp. 3325.
A. Arneodo, G. Grasseau, and M. Holschneider, Wavelet Transform of Multifractals, Phys. Rev. Lett., 61 (1988), pp. 2281.
A. Arneodo, Wavelet analysis of fractals: from the mathematical concepts to experimental reality, Wavelets. Theory and applications, Oxford Univ. Press, 1996.
M. Vergassola and U. Frisch, Wavelet transforms of self-similar processes, Physica D, 54 (1991), pp. 58.
I. Rodriguez-Iturbe and A. Rinaldo, Fractal River Basins: Chance and Self-Organization, Cambridge Univ. Press, New York, 1997.
J. Wang and R.L. Bras, A model of surface heat fluxes based on the theory of maximum entropy production, Water Resour. Res., 45 (2009), W11422.
J. Wang and R.L. Bras, A model of evapotranspiration based on the theory of maximum entropy production, Water Resour. Res., 47 (2011), XXXXXX.
M. Tribus, Thermostatics and thermodynamics; an introduction to energy, information and states of matter, with engineering applications, Princeton, N.J., Van Nostrand, 1961.
C.D Peters-Lidard, F. Pan, A.Y. Hsu, and P.E. O’Neill, ESTAR and model-derived multiscaling characteristics of soil moisture during SGP’97 Washita ‘92 and Washita ‘94, IEEE Proceedings, (2001), pp. 1297-1299. 25
Additional MaterialAdditional Material
27
128
64
3216
WV AnalysisWV Analysis
28
128
128
29
64
64
30
32
32
31
16
16
32
Daubechies, p=6
33
Haar
. Haar
. Daubechies, p=3,6
. Coiflet, p=1
. Symmlet, p=4
. Battle-Lemarie, p=3