open data portals: 9 solutions and how they compare
TRANSCRIPT
![Page 1: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/1.jpg)
Open Data Portals9 Solutions and How They Compare
![Page 2: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/2.jpg)
Dale
Lut
z
Stew
art H
arpe
r
![Page 3: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/3.jpg)
![Page 4: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/4.jpg)
![Page 5: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/5.jpg)
Why Open Data? And Why Now?
![Page 6: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/6.jpg)
Citizens are demanding access to open data like never before.
We want it.
![Page 7: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/7.jpg)
Great advances have been made in technology. Citizens are tech
savvy and aware.
We have the ability.
![Page 8: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/8.jpg)
Better decisions are being made by the mass population.
Benefits are undeniable.
![Page 9: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/9.jpg)
Why does this matter?
Build innovations on top of institutes and empower citizens across the globe.
![Page 10: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/10.jpg)
What is Open Data?
“Open data and content can be freely used, modified, and shared by anyone for any purpose”
opendefinition.org
“Open Data is free, public data that can be used to launch commercial and nonprofit ventures, do research, make data-driven decisions, and solve complex problems.”
Open Data 500
![Page 11: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/11.jpg)
Open Data Audiences● Citizens who want to examine the data
and answer questions they have.● Researchers and journalists who want to
gather and analyze the data to tell stories.● Developers who want to use the data to
build applications
Source: City of Winnipeg
![Page 12: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/12.jpg)
Who
Pro
duce
s Op
en D
ata?
All levels of governments
Intergovernmental Organizations
Non Governmental Organizations Private Companies
Academic Institutes
![Page 13: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/13.jpg)
Governments are doing better
![Page 14: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/14.jpg)
PDFs make for poor Open Data
spatialityblog.com/2011/08/31/mapping-hurricane-nyc/
![Page 15: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/15.jpg)
NYPD Example
Released collision data
in PDF
Citizen created a PDF Scraper Tool so the data could
be read
NYPD responded and their
data is now on Socrata
Limit public interaction
![Page 16: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/16.jpg)
Ingredients for Open Data Portal Success
![Page 17: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/17.jpg)
UpdatingDatasets should not be static and
should be regularly updated.
![Page 18: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/18.jpg)
How to keep your data up-to-date:
Provide your data as a published feed (e.g. RSS) or API.
Connect your open data platform directly to the master database.
Enlist the help of FME to sync your master data store with your open data repository.
![Page 19: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/19.jpg)
Coordinate SystemProjections
(Spatial Data Only)
![Page 20: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/20.jpg)
Projection Support● Advanced users should be able to choose their
projection. ● Local (e.g. State Plane or British National Grid)
and global projections should be provided. ● Globally we would recommend:
● WGS 84 Lat/Lng (EPSG: 4326)
● Spherical Mercator (EPSG: 3857)
![Page 21: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/21.jpg)
Ensure Data Quality
![Page 22: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/22.jpg)
Pay Attention To Data Quality● Ensure you’ve covered off our Data Quality
Checklist○ FME can automate the checking
● Check out csvlint.io -- a free CSV quality checker
![Page 23: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/23.jpg)
Delivery MechanismsFormats / APIs
![Page 24: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/24.jpg)
Data should be available in...
Tabular:● CSV● JSON (API)
Spatial: ● GeoJSON (API)● Shapefile
Tabular:● Excel● XML
Spatial: ● AutoCAD DXF/DWG● Esri File Geodatabase● GML● KML● Mapinfo TAB
Suggested: Supplementary:
![Page 25: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/25.jpg)
9 Solutionsfrom A-Z
![Page 26: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/26.jpg)
9 Solutionsfrom A-Z
ArcGIS Open DataAmazon Web Services (AWS)CKANDataPressDKANFTPGitHubJunarOpenDataSoft Socrata
![Page 27: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/27.jpg)
ArcGIS Open DataConfigure your own branded Open Data site
on top of ArcGIS Server or ArcGIS Online.
OPENDATA.ARCGIS.COM
![Page 28: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/28.jpg)
Popular choice for Esri users.Open data builds directly on top of your published ArcGIS services.
Supported data types:● Hosted (AGOL) feature
services● ArcGIS for Server
feature services● ArcGIS for Server map
services
● Image services● CSVs● Other - Web maps, URLs,
Word docs and PDF● Other services via Koop
![Page 29: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/29.jpg)
Data and metadata can be viewed in the browser.
Can create simple histograms, line, donut or scatter charts to look for patterns without downloading the data.
All data is downloaded in WGS 84, and you can also download the data as KML, Shapefile or via the API (JSON,
Geoservice, WMS).
![Page 30: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/30.jpg)
Demo
Sample AGOL Hosted Site
![Page 31: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/31.jpg)
Amazon Web ServicesRun/roll your own powerful open data platform.
AWS.AMAZON.COMwith help from WordPress & FMECLOUD.COM
![Page 32: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/32.jpg)
By leveraging the lower-level services of AWS (e.g. S3, EC2, RDS) and making use of FME Cloud as a data-mover, you
can produce an extremely fault-tolerant, scalable and powerful service that is
easy to maintain and cost effective.
![Page 33: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/33.jpg)
![Page 34: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/34.jpg)
Demo
Data Migration Project: GeoStor
![Page 35: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/35.jpg)
CKANOpen source data portal - providing tools to streamline publishing, sharing, finding and
using data.
CKAN.ORG
![Page 36: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/36.jpg)
Leading open source data portal with over 300 open source data management extensions. Particularly
suited for large organizations.
![Page 37: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/37.jpg)
Easy data uploading. Fast search experience.
A rich JSON API allows for integration with FME.
CKAN can plot geographic data in an interactive map.
Real World:The City of Surrey partnered FME Server to allow any CKAN dataset to be downloaded in any format and any projection.
![Page 38: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/38.jpg)
DataPressWordPress and CKAN seamlessly integrated
and hosted on the cloud.
DATAPRESS.IO
![Page 39: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/39.jpg)
Fast and simple data publishing
WordPress integration allows you to write blog posts, design pages, and manage menus and content in a
simple web interface
- Hosted on the cloud -
![Page 41: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/41.jpg)
DKANComplementary offering to CKAN. Open
source Drupal plugin with a cloud-hosted version available.
NUCIVIC.COM/DKAN
![Page 42: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/42.jpg)
Integrates CKAN features into DrupalSimple to deploy and maintain
Complete set of content management featuresEspecially suited to anyone using Drupal
Self-hosted or cloud version available
Real world: used by data.gov.uk
![Page 44: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/44.jpg)
Simple file catalog service.
Free hosting with sites like DataHub.io, FTP, Google Drive and GitHub is a good place to start with Open Data.
Cloud file storage solutions can be used to store and serve large volumes of data.
Web interface can be built on top of the storage system.
![Page 45: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/45.jpg)
Demo
FTP
![Page 46: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/46.jpg)
Demo
GitHub
![Page 47: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/47.jpg)
JunarCloud open data platform with focus on ease of use, powerful analysis and visualizations.
JUNAR.COM
![Page 48: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/48.jpg)
SaaS PlatformProvides a standard data portal as well as APIs that enable
developers to integrate data into their applications.
● Government of Chile● City of Sacramento● City of Palo Alto
Real World
![Page 49: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/49.jpg)
OpenDataSoftFocus on ease of use, automated API
generation and interactive visualizations.
OPENDATASOFT.COM
![Page 50: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/50.jpg)
Proficient working with large datasets, as
they leverage Elasticsearch, which ensures near real-time search and analysis.
SaaS platform
Geospatial format support
Data publishing and management via live dashboards
![Page 51: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/51.jpg)
SocrataA platform that turns data into a utility that can be discovered, consumed, visualized,
analyzed, and shared.
SOCRATA.COM
![Page 52: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/52.jpg)
Easy to publish data to Socrata using a WebUI, desktop sync tool, or API
Upload CSV, Excel, and TSV files natively with support for Shapefiles, KML, KMZ, and GeoJSON
There is both a published and working copy of the dataset
Many tools around metadata management and workflow
![Page 53: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/53.jpg)
Efficient search
Rich set of tools allow you to visually inspect data
Metadata is not front and center: the focus is on the information within the data itself
Impressive analytics toolsPublic Data as a Utility
![Page 54: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/54.jpg)
Demo
Socrata (Chicago)
![Page 55: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/55.jpg)
Solutions TakeawayThere are many top quality open data
solutions available. Do your own research and see which one is right for you.
![Page 56: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/56.jpg)
Hackathons (i.e. BigApps NYC 2015)
Normalization of data at national level so we can compare cities
globally
Open Private Sector
Focus shift: data quality, not quantity
More cities & governments following NY
The Future of Open Data
No tech excuses
![Page 58: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/58.jpg)
Questions? We have answers.
![Page 59: Open Data Portals: 9 Solutions and How they Compare](https://reader030.vdocument.in/reader030/viewer/2022020307/55bdf400bb61eb9a2d8b479d/html5/thumbnails/59.jpg)
Thank you
Want more resources about moving data?
blog.safe.comknowledge.safe.comwebinars.safe.com