web-based open source gis: decision support tools explaining the software stack presented by

Web-Based Open Source GIS:

Decision Support Tools Explaining the Software Stack

Presented by Aaron Racicot – GIS Programmer

[email protected] 19th, 2006

A Citizen of Salmon Nation

Outline

Introduction - Ecotrust Benefits/Limitations of Open Source OCEANSystem

Problem description Technology Overview - Software Stack

FORESTSystem Problem description Technology Overview - Software Stack

Where is it all going?

Who am I…B.S. Computer

Science

Split Personality

Telemark/BC SkiingRafting Guide Mountain Rescue

M.S. Environmental Science

Open Source User/Developer

GIS Programmer

Ecotrust - Salmon Nation

Benefits/Limitations of Open Source

Benefits Software cost = $0 Source code is available and modifiable User and development communities flourish Development cycles are VERY fast

Limitations Total cost is NOT = $0 Many applications are not as polished as the

proprietary counterparts Compatibility with proprietary software can be an

issue

Software Stacks… Desktop

Generic Stack FOSS4G Stack Grouping

Application QGis, Grass, OSSIM, JUMP, UDig

User Interface

Application Dev. Environment

Eclipse, QT, OpenGL

High Level Utilities GeoTools, PostGIS Data Serving

High Level Scripting Languages

PHP, PERL, Python

Low Level Utilities Shplib, GEOS, OGR/GDAL, PostGIS, R-Statistics, GMT

Data Processing

Low Level Languages

C, C++, Java, Fortran System Software

Operating System

Linux, Darwin, Cygwin

Hardware Drivers

32-64 bit processor drivers

Desktop Software Stack

Software Stacks... Server

Server Software StackGeneric Stack FOSS4G Stack Grouping

Client Side Browser Firefox, Safari, Netscape User Interface

Client Side Scripting

Java Script, Java Applets

~~~~~~~~~~~WEB~~~~~~~~~~~

Server Side Application

Chameleon, Cartoweb3, Custom Data Serving

High Level Utilities Mapserver, PCO, Grass

Low Level Utilities Shplib, GEOS, OGR/GDAL, PostGIS, R-Statistics, GMT

Data Processing


PHP, PERL, Python

Low Level Languages


Operating System/Drivers

Linux, Darwin, Cygwin

Hardware Drivers 32-64 bit processor drivers

Software Stack... DSTs (Ecotrust)

DST Software StackGeneric Stack FOSS4G Stack Grouping

Client Side Browser Firefox, Safari, Netscape User Interface

Client Side Scripting

Chameleon/Custom

~~~~~~~~~~~WEB~~~~~~~~~~~

Server Side Application

Chameleon Data Serving

High Level Utilities Mapserver, GRASS, PostGIS

Low Level Utilities Shplib, GEOS, R-Statistics, GMT, OGR/GDAL

Data Processing


PHP, PERL

Low Level Languages


Operating System/Drivers

Linux (Fedora)

Hardware Drivers 32 bit processor drivers

Work Flow – Real-Time Web DST

Web User Request

Web User Response

StaticData Storage

WebService

sData Gathering

Data Formatting

Data Processing

Map Formatting

Map Production

Real-Time Web Decision Support Tools

Work Flow – What OSGIS is good at

Web User Request

Web User Response

StaticData Storage

WebService

sData Gathering

Data Formatting

Data Processing

Map Formatting

Map Production

The Problem Is The Arrows! Connecting a web request to server side GIS analysis is tricky

Real-Time Web Decision Support Tools

Data Flow... DSTs (Ecotrust)

Raster based DSTSiuslaw Watershed Restoration Initiative

Data Gathering

Data Formatting

Data Processing

PHP Mapscript

GRASS

Raster based DST - Code

class Grass_GIS { var $data_dir; // Where the GRASS data

dir is var $rc_dir; // Where the GRASS rc dir is var $bin_dir; // Where the GRASS bin dir

is var $location; var $mapset; function Grass_GIS($data_dir,

$rc_dir, $bin_dir, $location, $mapset){

$this->data_dir = $data_dir; $this->rc_dir = $rc_dir; $this->bin_dir = $bin_dir; $this->location = $location; $this->mapset = $mapset; } function

Grass_GIS_copy($grass_gis){ $this->data_dir = $grass_gis-

>data_dir; $this->rc_dir = $grass_gis->rc_dir; $this->bin_dir = $grass_gis->bin_dir; $this->location = $grass_gis-

>location; $this->mapset = $grass_gis->mapset; } function set_location($location) {$this->location = $location;} function set_mapset($mapset) {$this->mapset = $mapset;} function set_data_dir($data_dir) {$this->data_dir = $data_dir;} function set_rc_dir($rc_dir) {$this->rc_dir = $rc_dir;} function set_bin_dir($bin_dir) {$this->bin_dir = $bin_dir;}

function run_command($cmd){ $std_output = ""; $grass_exec = ""; // Check that all the required info is available.. if ($this->data_dir != NULL && $this->rc_dir != NULL &&

$this->bin_dir != NULL && $this->location != NULL &&$this->mapset != NULL){// Here we will write out the script to run...$fp_data = fopen($this->data_dir."/grass_run.sh","w");// write out text headerfwrite($fp_data,"export GISBASE=".$this->bin_dir." \n");fwrite($fp_data,"export GISDBASE=".$this->data_dir." \n");fwrite($fp_data,"export GISRC=".$this->rc_dir."/.grassrc6 \n");fwrite($fp_data,"export ETC=".$this->bin_dir."/etc \n");fwrite($fp_data,"export PATH=".$this->bin_dir."/bin:".$this->bin_dir."/scripts:".$this->bin_dir. "/garden/bin:\$PATH:/usr/bin:/usr/local/bin/ \n");fwrite($fp_data,"export LOCATION_NAME=".$this->location." \n");fwrite($fp_data,"export MAPSET=".$this->mapset." \n");fwrite($fp_data,"export LOCATION=".$this->data_dir."/".$this->location."/".$mapset." \n");fwrite($fp_data,"\n");fwrite($fp_data,$cmd."\n");fflush($fp_data);fclose($fp_data);$fp_data = fopen($this->rc_dir."/.grassrc6","w");fwrite($fp_data,"GISDBASE: ".$this->data_dir."\n");fwrite($fp_data,"LOCATION_NAME: ".$this->location."\n");fwrite($fp_data,"MAPSET: ".$this->mapset."\n");fwrite($fp_data,"PAINTER: ppm\n");fwrite($fp_data,"MAPLP: stuff.ppm\n");fflush($fp_data);fclose($fp_data);$grass_exec = sprintf("chmod u+x ".$this->data_dir."/grass_run.sh");exec($grass_exec,$std_output,$std_error);

$grass_exec = sprintf($this->data_dir."/grass_run.sh");exec($grass_exec,$std_output,$std_error);

}

Raster based DST - Codedefine("GRASSDATA_DIR", "/var/www/html/apps/siuslaw_target/data/GRASSDATA/");$location_name = "OR_siuslaw“;$mapset_name = "land_targets“;$session_grass_dir = $session_tmp_dir."/GRASSDATA";

// Make a Dir to work inecho " Make a directory to work in inside this session ";$dir_exec = sprintf("mkdir ".$session_grass_dir);exec($dir_exec,$arr,$err1); $dir_exec = sprintf("mkdir ".$session_grass_dir."/rc");exec($dir_exec,$arr,$err1); $template_path = GRASSDATA_DIR."siuslaw_epa_template";echo " Copying over new GRASS GIS template ";$dir_exec = sprintf("cp -R ".$template_path." ".$session_grass_dir."/".$location_name);exec($dir_exec,$arr,$err1); // Make a MAPSETecho " Setting up the siuslaw specific area ";$dir_exec = sprintf("mkdir ".$ session_grass_dir."/".$location_name."/".$mapset_name);exec($dir_exec,$arr,$err1); // Copy over the projection info from the PERMANENT areaecho " Setting projections ";$dir_exec = sprintf("cp ".$template_path."/PERMANENT/PROJ* ".$ session_grass_dir."/".

$location_name."/".$mapset_name."/");exec($dir_exec,$arr,$err1); $grass_session = new Grass_GIS($session_grass_dir, $session_grass_dir."/rc", "/usr/local/grass-6.0.0", $location_name,

$mapset_name);$grass_session->run_command("g.list type=rast");

Vector based DST

Mapserver

PHP -> PHPMapscript -> PostGIS

PostGIS

Vector based DST - Code// Now connect to the db$host_str = "host=localhost";$db_str = "dbname=shelf_closure_db";$usr_str = "user=web_user";$string = $host_str." ".$db_str." ".$usr_str;$connection = pg_connect($string);if (!$connection){ pg_close($connection); echo "Error: Cannot connect to the database \n";} else { echo "Connected to the SCA database \n";}

// Create a dynamic VIEW in PostGISecho "* Create postgresql VIEW * \n";$exec_str = sprintf("CREATE VIEW ". $previous_session_name."_".$previous_session_id." AS ". "SELECT bk.the_geom, sum(tr.total_lbs*tb.percent) as total_lbs ". "FROM tow%02d_3kblk_intersect as tb, trawl%02d". " as tr, cablk3km as bk, port as pt ". "where tb.blk_gid=bk.gid ".$species_sql." ".$port_sql." ".$gear_sql. " and tb.tow%02d_id=tr.tow_id2 ". "GROUP BY bk.the_geom HAVING count(tr.tow_id2)>2", $start_year_short,$start_year_short,$start_year_short);$result_pg_exec = pg_exec($connection, $exec_str);if (!$result_pg_exec) { printf("%s \n", pg_errormessage());}

Vector based DST - Code

// Execute the pgsql2shp to create the outputecho "* Execute pgsql2shp create the shapefile in session directory * ";$dir_exec = sprintf("/usr/local/pgsql/bin/pgsql2shp -u web_user -f ". $session_tmp_dir."data/target_areas/test_output.shp shelf_closure_db ". "\"SELECT * from ".$previous_session_name."_".$previous_session_id."\"");exec($dir_exec,$arr,$err1); for ($ii=0;$ii<count($arr);$ii++){ echo $arr[$ii]." \n";} // Drop the dynamic table echo "* DROP the postgresql VIEW * \n";$result_pg_exec = pg_exec($connection, "DROP VIEW ". $previous_session_name."_".$previous_session_id);if (!$result_pg_exec) { printf("%s \n", pg_errormessage());}

13 Step Tool Install

1) First install Fedora Core4 with updates2) Ming

wget http://easynews.dl.sourceforge.net/sourceforge/ming/ming-0.3beta1.tar.gz

tar -xzvf ming-0.3beta1.tar.gz

cd ming-0.3beta1

make

make static

cd php_ext/

make

make install

echo "Don't forget to add 'extension=php_ming.so' to php.ini!"

3) Pdflib

wget http://www.pdflib.com/products/pdflib/download/602src/PDFlib-Lite-6.0.2.tar.gz

tar -xzvf PDFlib-Lite-6.0.2.tar.gz

cd PDFlib-Lite-6.0.2

./configure

make

make install

4) Update ldconfig

Add /usr/local/lib to /etc/ld.so.conf

Run : /sbin/ldconfig

5) Proj.4

tar -xzvf proj-4.4.9.tar.gz

cd proj-4.4.9

cd nad/

ls

cp ../../proj-nad27-1.2.tar.gz .

tar -xzvf proj-nad27-1.2.tar.gz

cd ..

./configure

make

make install

6) GDAL

Installed the ECW SDK (do a configure, make and install)

Installed the MrSid SDK (moved to /usr/local)

wget http://gdal.org/dl/gdal-1.3.1.tar.gz

tar -xzvf gdal-1.3.1.tar.gz

cd gdal-1.3.1

./configure --with-ecw --with mrsid=/usr/local/GeoExpressSDK/

make ogr-all

make install

which gdalinfo

7) GEOS

wget http://geos.refractions.net/geos-2.2.1.tar.bz2

tar -xjvf geos-2.2.1.tar.bz2

cd geos-2.2.1

./configure

make

make install

8) POSTGRESQL

tar -xzvf postgresql-8.1.2.tar.gz

cd postgresql-8.1.2

NOTE the LDFLAGS here... this is to support GEOS for postGIS

LDFLAGS=-lstdc++ ./configure --with-perl --prefix=/usr/local/pgsql_8_1_2

gmake

gmake install

Add the shared library path to /etc/ld.so.conf :

/usr/local/pgsql_8_1_2/lib

Run /sbin/ldconfig

Run make install in the ./doc directory to install the documentation

Finally add the following to the profile file to make paths available:

PATH=$PATH:/usr/local/pgsql/bin

MANPATH=$MANPATH:/usr/local/pgsql/man

export MANPATH

Add the postgis user

useradd postgis

passwd

[postgis@localhost pgsql]$ postmaster -D ./data/ > ./data/logfile.txt 2>&1 &

13 Step Tool Install9) POSTGIS

Just make sure that the Postgresql is configured with the LDFLAGS variable set

and that the GEOS software is downloaded and installed from Refractions.

./configure --with-proj --with-geos --with-pgsql

make

make install

Now we can create the database:

createdb -O aaronr test_db

createlang -U aaronr plpgsql test_db

psql -f /usr/local/pgsql_8_1_2/share/postgresql/contrib/lwpostgis.sql -d test_db

10) PHP

./configure --prefix=/usr/local/php4 --program-suffix=4 --enable-force-cgi-redirect --with-config-file-path=/etc/httpd/ --with-gd=/usr/local/ --with-jpeg --with-png --with-tiff --with-zlib --with-freetype-dir --without-ttf --with-mysql --with-regex=system --enable-dbase --enable-dbx --enable-versioning --with-pgsql=/usr/local/pgsql_8_1_2/

make

make install

strip sapi/cgi/php

cp sapi/cgi/php /var/www/cgi-bin/php4

cp php.ini-dist /etc/httpd/php.ini

Made the following mod to /etc/httpd/php.ini:

; Directory in which the loadable extensions (modules) reside.

;extension_dir = "./"

extension_dir = "/etc/httpd/php_mods"

mkdir /etc/httpd/php_mods

Added the following to /etc/httpd/conf/httpd.conf

#

# For PHP scripts as CGI-BIN

#

AddType application/x-httpd-php-cgi .php4 .phtml

Action application/x-httpd-php-cgi /cgi-bin/php4

Now re-start the server:

/etc/rc.d/init.d/httpd restart

Stopping httpd: [ OK ]

Starting httpd: [ OK ]

11) MAPSERVER

./configure --with-jpeg --with-gd --with-freetype --with-zlib --with-png --with-pdf --without-tiff --with-proj --with-threads --with-ogr --with-gdal --with-postgis --with-wfs --with-wmsclient --with-wfsclient --enable-debug --with-php=/src/php/php-4.4.2

make

cp legend mapserv scalebar /var/www/cgi-bin/

cp mapscript/php3/php_mapscript.so /etc/httpd/php_mods/

12) GRASS

CFLAGS="-g -Wall" ./configure --with-gdal=/usr/local/bin/gdal-config --with-postgres-includes=/usr/local/pgsql/include/ --with-postgres-libs=/usr/local/pgsql/lib/

make

make install

13) R-Statistics

tar -xzvf R-2.1.1.tar.gz

cd R-2.1.1

./configure

make

make install

Where to go for more infoOSGIS

Maptools - http://www.maptools.org FreeGIS - http://freegis.org/ Open Source GIS - http://opensourcegis.org/

Standards OGC - http://www.opengeospatial.org/

Desktop GRASS - http://grass.itc.it/ QGIS - http://qgis.org/ UDIG - http://udig.refractions.net/confluence/display/UDIG/Home JUMP – http://jump-project.org/ OpenEV - http://openev.sourceforge.net/

Server/Web Mapserver - http://mapserver.gis.umn.edu/ GRASS - http://grass.itc.it/ PostGIS - http://postgis.refractions.net/

Tools Remote Sensing - http://remotesensing.org/tiki-index.php GDAL/OGR - http://gdal.maptools.org/index.html PROJ.4 - http://proj.maptools.org/ R-Statistics - http://www.r-project.org/ GMT - http://gmt.soest.hawaii.edu/

The End

Tool Screen Shots Follow

Backup - GRASS

Backup – R-Statistics

Backup – PostGIS

Geometry

WKT Geometry

Backup - QGIS

web-based open source gis: decision support tools explaining the software stack presented by

Documents

pythonlow level languagesc

perllow level languagesc

grasslow level utilitiesshplib

web request

proprietary software

software stackpresented

open source gis

cygwinhardware drivers32