document: geotools 6.1 manual description: user and ... is a gis extension module for vtiger crm...

Document: GeoTools 6.1 Manual

Description: User and Administration Guide for installing and using GeoTools – The GIS extension module for vtiger CRM 6.1

Revision: 8

Document Status: Published

Written by: Alan Lord

Date Approved: 02/11/2015

© 2015 Libertus Solutions Revision No. 8

Table of ContentsIntroduction........................................................................................................................................................3User Guide...........................................................................................................................................................4

Using GeoTools.............................................................................................................................................4About Geocoding.................................................................................................................................................. 5Search parameters explained............................................................................................................................. 6Multi-coloured markers....................................................................................................................................... 7

Geocoding Accuracy....................................................................................................................................8Incorrect Geocodes............................................................................................................................................... 8Locking the Location.......................................................................................................................................... 10Non-geocoded records...................................................................................................................................... 12

Administration Guide.....................................................................................................................................14Installation...................................................................................................................................................14Configuring and Managing GeoTools....................................................................................................17

Core GeoTools Settings..................................................................................................................................... 18GeoTools Module Settings................................................................................................................................ 21Multi-coloured markers....................................................................................................................................... 23Background Scheduler....................................................................................................................................... 26Tips for Geocoding on shared hosts................................................................................................................ 26The Geocoding Process..................................................................................................................................... 27Geocoding in real-time...................................................................................................................................... 27

Important Information about GeoTools.....................................................................................................28


IntroductionWelcome to GeoTools from Libertus Solutions. GeoTools is the Geographic Information System (GIS)1 extension module for vtiger CRM 6.1.0.

GeoTools is a GIS extension module for vtiger CRM that offers users a completely new perspective with which to analyse and view their data.

Example uses of GeoTools:

● A salesman visiting a customer or prospect in a particular location could quickly identify other Leads or Organisations that are within an suitable radius of his appointment.

● Marketing could use GeoTools to visualise where their customers are located; for analysisor to target Leads in a particular Geographic area.

● Support teams could use GeoTools to research where service calls are being made or schedule preventative maintenance type visits with other customers when in a particular area.

1 A geographic information system (GIS) is a computer system designed to capture, store, manipulate, analyse, manage, and present all types of geographical data.


User Guide

Using GeoToolsThe user-facing side of GeoTools allows the viewer to search or filter records in the vtiger CRM using geographic parameters in addition to all the other filtering & search options vtiger has to offer. An active map is displayed alongside the list of records with each record also displayed on the map in its geographic location as a marker based on the chosen parameters of the search. The map itself can be zoomed in or out and can be dragged to change the area in view. If configured appropriately, GeoTools will display different coloured markers on the map. The colour of these markers will be generated automatically based on a selected Picklist for the module.

Full Screen map:You can also make the map itself go full screen by clicking theicon in the map:


About GeocodingGeoTools runs a background process once per day (or however frequently it has been configuredby the vtiger administrator) which does something called Geocoding1. It looks through every record in all configured modules, collects each record’s address information and tries to determine the geographic location of that record as a pair of Latitude and Longitude coordinates using an external Geocoding service.

If the Geocoding process can determine a location it then records these coordinates in the vtigerCRM database. This process can take some time (many days if you have a large number of records) depending on how many records are in your vtiger CRM system and the accuracy of your address data.

Once there is Geocoded data in the system, the user will be able to see these records displayed on the map and in the adjacent ListView. The user can click on the map icon or on the item in the list to find out which record they are looking at.

1 Geocoding is the process of finding associated geographic coordinates (often expressed as latitude and longitude) from other geographic data, such as street addresses, or ZIP codes (postal codes). With geographic coordinates the features can be mapped and entered into Geographic Information Systems, or the coordinates can be embedded into media such as digital photographs via geotagging. Source: Wikipedia


http://en.wikipedia.org/wiki/Geocoding

Search parameters explained

Displayed directly above the map are various parameters the user may edit to change the natureof the geographic search:

“Search In” allows the user to look at data from a different module. Any module that is configured for Geocoding in GeoTools will be visible in the picklist. Changing this option will reload the page.

The “Map Center” picklist determines which option to use to specify the central point of the map around which a radius search will be performed. The available options are:

● “Fixed Map Centre” - A location entered (as a pair of coordinates) by a vtiger Administrator in the Settings area

● “Company Details” - The Geocoded location of your Company Address (the Company Details are configured in the Administrator's area)

● “User’s Address” - Each user of vtiger can fill in their own address information in “My Preferences”. This option will only be visible if the current user’s address has already been successfully Geocoded

● “User’s Actual Location” - If selected, the user’s web browser will prompt for permission to allow GeoTools to try and determine the actual location in real-time. The accuracy of the detected location depends on a number of factors including browser, Internet access method and or availability of GPS services


● “From Other Entity Record” will bring up a popup window with a list of all Geocoded records for a particular module (that the user is permitted to view). The user can page through or search the list. Clicking on a single entry will then use that as the map’s central point

● “Manual Entry” enables the user to enter any address they like. If it can be geocoded successfully it will be used as the map centre

● There is one other option that is not visible until it has been triggered. A user can simply click and then drag the central icon around on the map to trigger a recalculation of the central location and a re-display of the data. The Picklist will subsequently display “Dragged Centre” when this option has been used

The “Search Radius” and “Unit“ options permit the adjustment of the radius distance parameters. For example, if there are no records within a 35Km radius of the chosen centre, the user may increase the value and/or change to Miles to recalculate the search results.

“Radius Constraint” instructs the GeoTools to search for records that are either Inside or Outside the boundary of the radius defined above.

Finally “Order By” tells the search tool to order the results either Nearest to the centre or Farthest away from the centre first.

The values selected for most of the options above will remain as the user chose until they navigate away from the GeoTools module or the user changes them.

Each of these values has a default setting which can be configured by a vtiger administrator.

Multi-coloured markersIf your administrator has configured it, GeoTools will show a Key, or Legend, above the map indicating which marker colour represents which value of the configured picklist. In the above screenshot, the Picklist configured is “Lead Source” and on this particular page of search results there are multiple values of Lead Source records on display. The Legend will automatically display only those values that are present in the search results. To help with identification, the checkbox on the left column of the List is also coloured.

The colour of these markers is determined automatically by the text value of the picklist field.

If a record doesn't have a value saved for the Picklist, then the default marker colour for that module will be used.


Geocoding Accuracy

It is important to recognise that Geocoding is not a precise science. Address formatting, punctuation, unusual spellings, lack of data for certain geographies etc. may all have an effect on the accuracy of the Geocoding process.

This section and the next offer some suggestions as to how to identify and then rectify any non or mis-Geocoded records.

Incorrect GeocodesIt can be useful from time-to-time to examine your data and look for incorrectly Geocoded records. These records will not show up in the Non-Geocoded list (see below) but they may also not appear in your searches if they have incorrect coordinates that put them outside of your search Radius.

Selecting the Outside option of the Radius Constraint and then choosing the Farthest 1st option of the Order By setting may help to locate records that should be in a user's searches buthave been incorrectly Geocoded:

In the above image there is a marker somewhere over Central America, but the Listview name and Contact details suggest this record should probably be in the UK.

Edit Address:If it is incorrectly Geocoded the first option would be to simply examine that record and edit the address details to try and get a more accurate Geocode the next time the process runs.


Drag Marker to recode:Another method of re-locating misplaced records is to actually drag the marker around the map

and then save its current location. This is possible by clicking on the the small “Edit Location” buttonthat is visible on the marker’s popup. Click on the marker you want to move to reveal the popup.

Once clicked, the Edit location button is replacedand will say instead “Save Location”. Before yousave the new location you can now drag the markeraround the map. Zoom out to drag longerdistances then zoom back in to locate the markeraccurately. The marker may be dragged as manytimes as the user needs to get it to the correctplace on the map. Once in the correct location,click the marker again to reveal the popup and clickthe “Save Location” button.

The record’s new location is now stored in thedatabase and will not be updated by any subsequent runs of the Geocoding scheduled job.


Locking the LocationIt is now possible to lock, and unlock the location information.

The Locked checkbox on each map marker will indicate if the record's location information will be updated or not. If the checkbox is not checked and the record's address information is edited then GeoTools will attempt to re-geocode the record's location as soon as it is saved.


Manual Entry testing:You can also use the Manual Entry option of the Map Centre picklist to “test” an address’ ability to be Geocoded. Enter the desired address details and GeoTools will try and Geocode this on-the-fly. If successful the map’s centre icon should then appear in the desired location. If it fails toGeocode the address, try it with less (yes less) information.


Non-geocoded recordsSometimes records will stubbornly refuse to Geocode at all. If records do not have sufficient address information, or the Geocoder can't understand the address data (which may sometimes happen), they will not be Geocoded at all. To assist the user in identifying these records GeoTools inserts a Custom Filter entry into the GeoTools ListView Filter options:

Choosing the Non-Geocoded option from the filter will list all records that have not been Geocoded for the module selected in the “Search In” picklist.

The user may then examine these records and edit or improve the address data so they can be Geocoded again the next time the scheduled Geocoding job runs.

Add New records to map manually:Alternatively a user can also add records from this non-Geocoded ListView straight onto the map and then edit this record's position by dragging the new marker to the correct location using the same method described in previous section Incorrect Geocodes. Multiple markers fromthe ListView may be added to the map but be aware they will all be “on top of each other” on the centre point of the map.


If necessary use the horizontal scrollbars to go the right hand side of the ListView row where the Distance column is located. Hover over an entry you want to add to the map and click on the little world icon that appears:

After clicking on the world icon a new marker will be placed in the centre of the map, whereupon the user can edit this new marker’s location as described in the previous section.

If the user decides they do not want to add a marker to the GeoTools database at this time, refreshing the page before saving the marker's location will remove it from the map.


Administration GuideThis section is primarily intended for the administrators of the vtiger CRM system but other users might find the content interesting or useful.

InstallationGeoTools is installed via the Extension Store which is an application within vtiger itself. This ensures that the version of GeoTools installed has been approved and tested by the vtiger team.The Extension Store also informs the administrator when updates are available, and enables easy upgrading, for any installed modules.

To install Geotools navigate to the CRM Settings area and click the button on the left column called “Extension Store”. This will display the various optional extension modules that are currently available in the vtiger Marketplace1. The extension modules in the Marketplace have been approved and tested by vtiger before being allowed to be installed by end users.

If this is the first time you have visited the Extension Store you will need to register a new user...

1 https://marketplace.vtiger.com


and you may also need to supply a credit card to allow the purchase of any non-free extension modules.

Once you have provided the necessary information you can either browse or search for the GeoTools module. Once found click on the “More Details” button to view more information about the module. If you wish to install GeoTools you should pay the requested amount via your credit card and then you will be able to install the module simply by clicking on the green “Install” button.


Configuring and Managing GeoTools

Once installed, GeoTools needs to be configured before it can be effectively used. GeoTools provides two separate configuration pages, one for the Core default parameters and a separate area to manage the other modules' configuration information. These options are accessible from the Module Manager in the CRM Settings area.

The default parameters are configured in the “Core GeoTools Settings” area and control all of the basic settings.

The Module configuration area is called “GeoTools Module Settings” and exists because GeoTools needs to know about each entity module the administrator decides should be includedboth for Geocoding and for user searching. The administrator must determine the fields in each of these modules that will be used to provide the address data to the Geocoding Scheduler.


Core GeoTools SettingsWhen GeoTools is first installed some default settings will be set and others will be left empty. The first screen in the Core Settings is a read-only display of the various parameters as they are currently configured.

Clicking the Edit button (top right) will open a new screen to enable editing of most of the parameters:

The first setting "Geocoding Service Provider” allows the administrator to change the provider of the Geocoding and the Map Tile service. Currently the only choice is OpenStreetMap.

GeoTools can already support Google’s Map and Geocoding services, but Libertus Solutions believes that using GeoTools with Google’s services requires a specific license. Please contact Libertus Solutions for further information if you are interested to use Google’s services within GeoTools.

The “Maximum Geocodes” parameter is an Integer that represents the maximum number of records that the GeoTools Geocoding Scheduler will attempt to Geocode in one run of the scheduler. The default is considered to be reasonable for the free Geocoding service provided by OpenStreepMap and Nominatim servers. Be aware that if you increase this significantly you may end up getting blocked.


“Default Radius” is the figure that is used when a user first opens the GeoTools view. This is an integer value. It specifies the radius within which entity records will be searched.

“Map Zoom” is the default level of zoom for the map display. This provides a level of consistencyfor the user although the map will automatically zoom to the appropriate level to display the results if they do not fit within that zoom level.

“Fixed Map Centre (lat, lng)” allows the administrator to enter a specific location as a latitude and longitude value that is not obtainable by any other method. This might be useful for companies where multiple offices exist and there is a need for a different location to the Geocoded Company Address. The format should be specifically: latitude,longitude

The “Distance Units” setting enables the administrator to determine the default unit of measurement for the Radius specified earlier. The options currently are either Miles or Kilometres.

The option “Default Module” sets the module that GeoTools will search when a user opens the GeoTools view for the first time. The administrator may choose any module that has already been configured in the GeoTools Module Settings area. If no modules have yet been configured this option will be empty.

The “Centre Map Marker Icon” is that which will be used to indicate the centre point on the GeoTools map during searches.

The field “Geocoded location of Company Details (lat,lng)” stores the latitude and longitude values either of Geocoding the address stored in the Company Details setting module, or manually entering the coordinates. If you change the address of the Company Details there is a button next to the field which will attempt to Geocode the new address and populate the field. Alternatively the latitude and longitude may be entered manually.


The “Default Map Centre” parameter determines which centre location will be used when GeoTools is first viewed by a user. The options mean the following:

● Fixed Map Centre (the location specified in the Fixed Map Centre field)● Company Details (the location of the Geocoded Company Details address)● User’s Address (each user of vtiger can fill in their own address information. If this value is

chosen and the user’s address has been successfully Geocoded this will be used as their initial central point when opening the GeoTools view)

● Browser Location (If selected, when a user visits the GeoTools page, it will prompt them to allow GeoTools to try and determine their actual location in real-time. The accuracy of the detected location depends on a number of factors including their browser, Internet access method and or GPS services)

Please be aware that these are the default settings. The User can change most of these parameters at any time during their use of the GeoTools module.


GeoTools Module SettingsThis Settings area is where the administrator defines which modules are to be Geocoded and made available to users for searching. The fields the Geocoder is to use for address data are selected here and there is also a facility to add and remove “Action Links” on the modules to enable rapid search functionality.

Any regular entity modules, custom entity modules, and the vtiger Users module may be configured for use in GeoTools. If the records of a particular module do not contain any useful address information however, the attempts at Geocoding will be a waste of time and resources.

Adding a new module is achieved by clicking the “Add Module” button:


This reveals a popup window with a single option picklist showing all suitable modules. Selectingone and clicking save will add it to the page.

The next step is to review and select the appropriate address fields to be used for this module. GeoTools expects fields to be configured for Street, City, County, Postcode and Country values. In some countries one or more fields will not be relevant and may be left as “Not Set”. But be aware that the accuracy and efficacy of the Geocoding process depends on the address data being supplied to it. Often it’s actually the Street field contents that can confuse the Geocoder. If your results are not very accurate or many records are not being Geocoded at all, try setting the Street field to “Not Set” and wait for the Geocoding Scheduler to run again.


Multi-coloured markersFor each module the administrator should also decide on the colour of the “default” marker to display on the map for this module’s data. Then below this, if the module contains any Picklists, it is also possible to choose a single picklist to use to show different colours of marker on the map.

In the above example, the Picklist “Lead Source” is being chosen for the Leads module configuration. When viewing search results on the GeoTools map, if the Lead Source field of any records being displayed contains a value, the Lead marker's icon will change colour and there will be a Key shown directly above the map to help identify which value is represented by which colour.

The colour of these markers is determined automatically by the text value of the field in each record; it can not be user-configured. The advantage of this approach is there is no additional“management” required if the Administrator needs to add, delete or change values of thePicklists.

Click “Save” at the top of the GeoTools Module Settings page to save this information.

Within each Module’s configuration table there are several buttons and alongside the Module’s name some useful information indicating how many records there are in total and how many have been successfully Geocoded.


The “Add Action Link” button enables the Administrator to add a link to this module so that when a user is looking a Detail or Summary view of a particular record, they can clickthis Action Link to take them to the GeoTools map using thisrecord as the Map’s central point.

The administrator may enter a different Label for the link and may optionally choose a target module to perform the initial radius search against. If left unchanged it will use the Default Module configured in the Core GeoTools Settings. The administrator can now also choose a Filter to use in the target module. This could be used, for example, if you wanted to locate all “Hot Leads” within a radius of a Contact you are going to visit in the near future.

The “Remove Action Link” button allows the administrator to remove a link from this module byselecting it by the Label from the picklist.


The “Delete Module” button will remove the configured module from the GeoTools database and will also remove any associated action links.


Background Scheduler

Important Note: The vtiger scheduler only works if the vtiger cron feature has already been configured.

When GeoTools is first installed, a “scheduled job” is registered in the vtiger CRM scheduler to run once every 24 hours.

On initial installation of GeoTools this scheduler is inactive. This should be set to active only after the GeoTools Module and Core Configuration settings have been setup as desired and described above.

Once GeoTools has been configured, edit the scheduler entry and set the Status to “Active”

The frequency of the Geocoding process may be configured here and the service disabled altogether if necessary.

Tips for Geocoding on shared hostsIf you have vtiger and GeoTools installed on a shared hosting service, you will almost certainly need to fine tune the parameters for Geocoding. We have discovered that some shared hosting services have very restrictive limits on the length of time a cron job can run for; this can be as short as just one or two minutes before the script will be terminated by the hosting service!

To ensure the Geocoding scheduled job will work properly, you will need to reduce the parameter “Maximum Geocodes” in the GeoTools Core Settings area to something quite small, e.g. “50”. Then you can increase the frequency of the scheduler so that it runs perhaps every 30 minutes or 1 hour instead of once per day. DO NOT try and configure GeoTools so it generates excessive geocoding requests to OpenStreetMap. You may well get all access blocked!


The Geocoding ProcessIn order of their configuration in the GeoTools Module Settings, the Geocoding scheduler builds a list of each configured module. For each module GeoTools then gets any new records, i.e. onesthat have not yet been Geocoded before. It then attempts to Geocode as many of these records as possible up to the “Maximum Geocodes” setting. If this completes and the “Maximum Geocodes” limit has not been reached, it then looks for any records that have been modified since the last time they were Geocoded and will then attempt to Geocode those. Finally, it looks for records that have been really deleted, i.e. removed from the recycle bin, and removes any corresponding Geocoded entries from the GeoTools database.

This background geocoding process will only try to Geocode a record once. If it fails to get a suitableresult it will mark that record as such and not try to Geocode it again unless something has changed(it's modified time is later than the last Geocoded time). For example, if it only managed to Geocode700 from 2500 Leads the first time and the rest failed they won't try to be Geocoded again until something changes in each record.

Geocoding in real-timeSince version 1.0.6 GeoTools also attempts to Geocode records in-real-time as well as using the frequency-based scheduler.

Each time a new record (for a module configured for Geocoding) is saved in vtiger, an vtiger Event Handler1will attempt to Geocode the record in the background if there is sufficient address information.

Each time an already Geocoded record is edited and saved, if any of the configured address fields have changed and the record is not locked, then the Event Handler will also attempt to Geocode and update the location information. GeoTools now supports the ability to lock and unlock the location information of each record by a checkbox shown on the Map Marker:

1


Important Information about GeoToolsThis release of GeoTools uses two free services from OpenStreetMap1:

● A Tile server to deliver the map images, and ● Nominatim, which Geocodes normal address information into a pair of Latitude and

Longitude coordinates.

OpenStreetMap is a project that creates and distributes free geographic data for the world. We started it because most maps you think of as free actually have legal or technical restrictions on their use, holding back people from using them in creative, productive, or unexpected ways.

Please be aware that both of these services are free to use and are based on Open Data. The capacity of the infrastructure to deliver these services is not unlimited. Consequently any overly aggressive use of either service may result in your access being blocked.

Libertus has intentionally limited the amount of records that can be Geocoded in any single day and also incorporates a small delay between each Geocoding request. Commercial Geocoding and Map Tile providers may offer faster or even unlimited services.

Libertus wrote GeoTools to be provider agnostic. That is to say, GeoTools can use other Geocoding and/or Tile server services from commercial or other providers such as Google or Mapbox for example. The use of other providers’ services typically requires the purchase of a license to use the service and some minor software development on GeoTools to use their particular APIs (Application Programming Interfaces).

If you wish to use GeoTools with an alternative Mapping and/or Geocoding provider please contact Libertus Solutions to discuss your requirements.

1 http://www.openstreetmap.org


http://www.openstreetmap.org/copyright

document: geotools 6.1 manual description: user and ... is a gis extension module for vtiger crm...

Documents