Download - Geoprocesamiento en Amazon
-
8/13/2019 Geoprocesamiento en Amazon
1/37
GEO PROCESAMIENTO EN WEB
-
8/13/2019 Geoprocesamiento en Amazon
2/37
Introduccin
Sources images:alkoholikuin.blogspot.comnro.nao.ac.jp
Informacin Geogrfica
publicacion
Internet
ProcessingDesktop applications
UploadDownload
Cloud computingProcessing
publicacion
Escenario A
C h
a n g
i n g
Escenario B
-
8/13/2019 Geoprocesamiento en Amazon
3/37
Evolucin Geopro
(Brauner et al., 2009)
Geoprocessing
Web Processing Services(WPS)
Improved WPS
Llevar los servicios donde se encuentra lainformacin (moving code paradigm)
Utilizar la tecnologia Grid Computing
Incorporar Service Oriented Architecture(SOA) caracteristicas
Parallelized WPSOn
Cloud Computing
-
8/13/2019 Geoprocesamiento en Amazon
4/37
Web Services Framework Of OGC GeoprocessingStandards
Web Processing Services (WPS)
-
8/13/2019 Geoprocesamiento en Amazon
5/37
Web Processing Services (WPS)
Interface estndar que permite enmascarar procesos,algoritmos y operaciones en la Web de una formadefinida y estructurada para ser encontrados y usados porclientes y otros procesos
WPS
GetCapabilities
GetDescription
Execute
Disponible servicios
Informacin acerca del servicio
Ejecuta
Current version 1.0.0
-
8/13/2019 Geoprocesamiento en Amazon
6/37
WPS Frameworks
Deegree
52North
PyWPS
Zoo
Extensions Grid Capabilities
GridGain
UNICORE
-
8/13/2019 Geoprocesamiento en Amazon
7/37
Cloud Computing
-
8/13/2019 Geoprocesamiento en Amazon
8/37
Definicin
Cloud computing is a technology that uses the Internet and
central remote servers to maintain data and applicationsWikiInvest
-
8/13/2019 Geoprocesamiento en Amazon
9/37
-
8/13/2019 Geoprocesamiento en Amazon
10/37
Proveedores de Cloud Computing
Amazon Web Service (AWS)Google App EngineMicrosoft AzureGoGridRackspace
o Amazon Elastic Compute Cloud(EC2)
o Elastic Block Store (EBS)o Multiple Locationso Elastic IP Addresso Auto Scalingo Elastic Load Balancingo Amazon Simple Storage Service
(S3)o Amazon CloudFronto Amazon Relational Database
Service
-
8/13/2019 Geoprocesamiento en Amazon
11/37
Aplicaciones Geospaciales en la Web
Publicaciones de Web Map Services (WMS) on Google App Engine(GAE)Implementacin de indices utilizando Hadoop technology sobre 110-millones de parcelas en una nube privadaGoogle maps utiliza cloud computing para soportar miles de usuarios
WPS service en AWS.. =>
-
8/13/2019 Geoprocesamiento en Amazon
12/37
WPS en Grid
-
8/13/2019 Geoprocesamiento en Amazon
13/37
-
8/13/2019 Geoprocesamiento en Amazon
14/37
El principio de los mtodos
o Dependencia de los vecinoso Proceso estacionario
Cmo es obtiene s depende del metodo
Kriging => Ordinary KrigingUniversal Kriging
Radial Basis Function (RBF)
Inverse Distance Weight (IDW)
-
8/13/2019 Geoprocesamiento en Amazon
15/37
Methods
InverseDistance Weight
(IDW)
Ordinaryand
UniversalKriging
Radial BasisFunction
(RBF)
Semivariogram Fitting theempirical toTheoreticalmodel.
LinearExponentialSpherical..,
Ordinary Kriging
Universal Kriging andits variance s 2
k power of interpolation
221 cr r
22
2
1
cr
r
Multiquadric :
Inverse multiquadric :
Thin plate spline :Multilog :Natural cubic spline :Spline with tension :
( r i) is the radial basis function
z is the evaluationof all points in the functionused
Preprocessing
Least Squares
Least Squares
None
CalculatingGetting s
-
8/13/2019 Geoprocesamiento en Amazon
16/37
Web Processing Services en Grid
Interpolacin
Caso de estudio
-
8/13/2019 Geoprocesamiento en Amazon
17/37
Cross Validation
IDW
Kriging(Ordinary andUniversal)
RBF
Number of neighborhoods from 5 to 12, andthe power value change from 0.1 to 10 atsteps of size 0.1.
Number of neighborhoods from 5 to 12,number of lags (7-15), models: Linear,Spherical and Exponential
Number of neighborhoods from 5 to 12, thesmooth factor from 0.1 to 0.5 at steps of size0.1 and sub models (Multiquadric, Inversemultiquadric, Thin plane spline, Multilog,Natural cubic spline)
-
8/13/2019 Geoprocesamiento en Amazon
18/37
Distribucin del proceso deinterpolacin
Pixels to beinterpolatedby each node
Points to besent to eachnode
-
8/13/2019 Geoprocesamiento en Amazon
19/37
Libreria para paralelizar procesos
Classes with
parallelizationfeatures
-
8/13/2019 Geoprocesamiento en Amazon
20/37
Adicionar servicios en paralelo
Profile by default in the52North Framework
-
8/13/2019 Geoprocesamiento en Amazon
21/37
Estructura de usada
Tomcat configuration Master Node (GridGain Node)
Master Node
Nodes
-
8/13/2019 Geoprocesamiento en Amazon
22/37
WPS services parallelized
Entradas:Data: WFS with GML and SHP-ZIP formatField: Contain the attribute to do the
interpolationMethod: Ordinary Kriging, UniversalKriging, IDW, RBF
Salida:RMS: Error of the best method in thecross validationStdRMS: Standardized ErrorCorrelation coefficient
Parameters: Parameters found.Cross Validation Graph: URL with crossvalidation graphFitting graph: Show the error behavioraccording to the method selectedIteration: URL with a log file with theiterations summary.
Entrada:Data: WFS with GML and SHP-ZIP formatField: Contain the attribute to dointerpolationMethod: This input receives astring with the method andparameters for executing theinterpolationResolution: Spatial Resolution
Salida:Result : WMS with reference tocoverage on GeoserverDuration: process duration
Validacin cruzada Interpolacin
-
8/13/2019 Geoprocesamiento en Amazon
23/37
Instalando a WPS en Amazon AWS(Linux)
Crear una cuenta enAmazon AWShttp://aws.amazon.com/account/
Instalar AWS service APIObtener credenciales variables (Linux)Crear una instancia
http://aws.amazon.com/account/http://aws.amazon.com/account/ -
8/13/2019 Geoprocesamiento en Amazon
24/37
Consola de Amazon AWS
Amazon AWS tiene 2433 instancias publicas
-
8/13/2019 Geoprocesamiento en Amazon
25/37
Utilizando 10 instancias
-
8/13/2019 Geoprocesamiento en Amazon
26/37
-
8/13/2019 Geoprocesamiento en Amazon
27/37
Informacin usada
[1] http://www.aemet.es/es/servidor-datos/acceso-datos/listado-contenidos/detalles/datos_observacion
Meteorological Agency of Spain (AEMET).
-
8/13/2019 Geoprocesamiento en Amazon
28/37
Datos de elevacin
The DEM is published by OpenTopo,created in 2008 with LIDAR, resolution of 0.5 meters,Universal Traversal Mercator (UTM) region 12 North,datum WGS84
Evaluacin con 1000 y 10000puntos
-
8/13/2019 Geoprocesamiento en Amazon
29/37
-
8/13/2019 Geoprocesamiento en Amazon
30/37
Software y hardware AWS
Master NodeSoftware
AMI Operating system fedoraTomcat 6.Java SDK 1.6.0.20.GridGain 2.1.1Geoserver 2.252North WPS Framework RC652North WPS OpenLayer client
Hardware simulated (1 micro Instance )Micro Instance 613 MB of memory 32-bitplatform
Hardware simulatedHigh-CPU Medium Instance 1.7 GB ofmemory, 5 EC2 Compute Units (2 virtualcores with 2.5 EC2 Compute Units each),350 GB of local instance storage, 32-bitplatform
Nine Nodes
Software
AMI Operating system fedora
Tomcat 6.
Java SDK 1.6.0.20.
GridGain 2.1.1
Hardware simulated (1 micro Instance )
Micro Instance 613 MB of memory 32-bit platform
-
8/13/2019 Geoprocesamiento en Amazon
31/37
Evaluacin de la libreria
y = 1.0075x
0
2
4
6
8
10
12
14
0 5 10 15
V a l u e I n t e r p o l a t e d
b y W P S
( G e o s t a t
i s t c a l
l i b r a r y
Value Interpolated by WPS (Geostatistcallibrary)
Comparison between Geos. libraryand ArcGIS Method Kriging
y = 0.9856x
0
2
4
6
8
10
1214
0 5 10 15
V a l u e
I n t e r p o l a t e d
b y W P S
( G e o s t a t
i s t c a l
l i b r a r y
Value Interpolated by Geostatisticalextension ArcGIS
Comparison between Geos. libraryand ArcGIS Kriging Univeral
Method
y = 0.9994x
0
2
4
6
8
10
12
14
0 5 10 15
V a l u e
I n t e r p o l a t e d
b y W
P S
( G e o s t a t
i s t c a l
l i b r a r y
Value Interpolated by Geostatisticalextension ArcGIS
Comparison between Geos. library andArcGIS
IDW method
y = 0.9994x
0
2
4
6
8
10
1214
0 5 10 15
V a l u e
I n t e r p o l a t e d
b y W
P S
( G e o s t a t
i s t c a l
l i b r a r y
Value Interpolated by Geostatisticalextension ArcGIS)
Comparison between Geos. library andArcGIS
RBF Multiquadratic Method
MethodGeostatistical Library ArcGIS (Geostatistical Extension)
RMSStandardized
RMS RMSStandardized
RMS
IDW1.607 - 1.607 -
Power=2
Ordinary Kriging1.587 0.881 1.605 0.829 Exponential(R:261138;S:5.4;N:
1.97)
Universal Kriging1.593 0.873 1.623 0.820 Exponential(R:29138;S:4.4;N:1.
67)
RBF1.648 - 1.648 - Model: Multiquadratic;
Factor=0
-
8/13/2019 Geoprocesamiento en Amazon
32/37
Comparasion interpolacin
RBF Kriging Ordinario
Small differences
-
8/13/2019 Geoprocesamiento en Amazon
33/37
Evaluation in WPS OpenLayer client
CrossValidation WPS service
Interpolation WPS service
-
8/13/2019 Geoprocesamiento en Amazon
34/37
1 2 3 4 1 2 3 4 1 2 3 4
Res: 2 Res: 5 Res: 10IDW 34686 28323 18286 17968 7152 5833 4497 4460 3688 4037 4391 2820Kriging 26132 23232 18487 14955 4464 4676 4289 3748 3414 2401 2602 2088
KrigingUniversal 34588 28482 21773 18639 7116 5742 4511 4593 3284 2856 2592 2586Spline 35908 26000 16980 19056 7389 5590 4290 4609 3760 4298 4391 2920
0
5000
10000
15000
20000
25000
30000
35000
40000
D u r a t
i o n
I n t e r p o l a t
i o n
M i l l i s e c o n d s
Utilizando intranet 4 nucleos(4 nodos, 1.000 points)
1.000.000 pixels 160.000 pixels 40.000 pixels
-
8/13/2019 Geoprocesamiento en Amazon
35/37
-
8/13/2019 Geoprocesamiento en Amazon
36/37
Datos a evaluar
Relacin numero de nodos y tiempo de interpolacin
Entorno de evaluacin:
Evaluacion: 2011-02-22 T 9:00:00 Z - 2011-02-22 T22:00:00 ZNumero de puntos: 10000Resolucin: 2 meters (1000000 pixels), 5 m (160.000pixels) and10 m (40.000 pixels).Numero de nodos(micro instances): 1-10Repeticiones: 10Solicitudes secuenciales 15 segundos
-
8/13/2019 Geoprocesamiento en Amazon
37/37
Resultados de la evaluacin
Method ResolutionNumberof Pixels
MaxDuration
Numberof Nodes
MaxDifference
PercentageDifference
Kriging 2 1000000 265467 10 -221610(a) -83%
Kriging 5 160000 14295 7 -10628(a) -74%
Kriging 10 40000 3311 8 -1136(a) -34%
IDW 2 1000000 227569 10 -187858(a) -83%
IDW 5 160000 11435 10 -8024(a) -70%
IDW 10 40000 2848 9 -713(a) -25%
Method ResolutionNumber of
PixelsMax
Duration
After thisnode (Notfind signif.
p-value