stad lier: transforming raw data into business info
TRANSCRIPT
Transforming Raw Data
into Business Information
Luc Janssens
Data-, -Analysis- & GIS-coördinator
Department Managementconsulting & Projectmanagement
City of Lier, Belgium
1: Data, Information & intelligence
Operational Environment
– Non- or thematically structured data
Data
– Structured collection of data
Information
– Structured presentation of data
Intelligence
– Data within a business / mutual context
Operational environment
= Non- or thematically structured data
Knowledge – What people know
– Dataflow model (creation and maintenance)
– BPMN as notation formalise processes
– Empowerment of the specialist in order to make his expertise accessible and reusable
Cabinets – Clean-up, digitize, archive
– Create re-usable indexes (meta-data) or digitize paper documents into reusable repositories.
Computers – A mix of formats, applications, open or closed, structured or
plain garbage.
– External & Internal sources, applications, acces-rights
– Quality = timely, complete, accurate, consistent, well-defined, unique
– Meta-Data describes data-quality
Data
= Structured collection of data
Data-Collection – Collect data from different sources
Data-Processing – Processing: Convert data into the desired data-model
– Cleaning: Erroneous, irrelevant, redundant or incomplete data
– Exploitation: Publish data for the desired target systems
Data-Needs – WHAT… do we want to manage, analyse, report upon, alarm upon,…
– In which form should we do this?
– Business WareHouse: Unique data-model for reporting
– MasterData: Specific model for cross application exchange
– Location WareHouse: Add location based intelligence
Information = Structured presentation, reporting en distribution
of data.
Information – Who, What, Where, When, Why and How?
– In what form? Automatied, distribution to other systems and/or organisations, BI, Reporting, Paper-output, GIS, …
Analysis & Production – Descriptive/post/pre-emptive statistics as automated intelligence
– Data-visualisation en data-exploration as manual intelligence
Intelligence – Data in a mutual context
– Focus on the essential within a specific context
– Collaborate within a specific context
– Alarming & Dashboards
2: Techniques & Products.
Technological choise
BPMN
ETL
RDBMS
BI
BI Reporting
QlikView & GeoQlik
nPrinting
Bizagi
FME
PostGreSQL & PostGIS
Statistieken
Meta-Data
1 Central Data-Warehouse (Business – MasterData – Location)
Central Data-Warehouse
PostGreSQL PostGIS
www.postgresql.org www.postgis.net
Data-Analysis / Dashboards / Collaboration
QlikView GeoQlik
www.qlik.com www.geoqlik.com
Data-Reporting / Distribution
QlikView nPrinting
www.qlik.com www.qlik.com
3: FME Usage and advantages.
Scripting / Shapeloader / SQL / …
– Meerdere omgevingen goed kennen
– Geen integratie / “Cowboy-code”
Selection (Comparision of 13 Products)
– Product A: Perfect for GIS… only for GIS
– Product B: … not useable for GIS
– Product C: … not useable for CAD
– Product D: … not useable for Office
– Product E: … more cryptical than scripts
– Product F: … no support
– Product G: … since 2013 no new releases
FME
– The Only Perfect Match!
ETL Selection
FME !
Datasets: – CAD / GIS / Raster / Data / Services
Applications: – 18 Suppliers / 46 Applications / > 2500 Datasets
Integration: – From “Hairball-Interfaces” to “Data-Integration-Plan”:
– Clear, manageable, documented, expandable interfaces.
Data - Quality: – Corrections, integrity, quality control
Data - Enhancements: – Combined, enhanced, scheduled data-sets
Data - Distribution: – Data in the right form on the right place (distributed / push / pull)
– Business Intelligence / Context Intelligence / Location Intelligence
FME - Modellen: – Even non-IT-personnel is able to model in FME
– Model = documentation
– Automatic en Manual workflows
FME in Lier
FME: The right data in the right form on the right place!
Conclusion: FME rocks!
Thank You!
Luc Janssens
• Data-, -Analysis- & GIS-coördinator
• Department Managementconsulting & Projectmanagement
• City of Lier, Belgium