geoindexing with mongodb
DESCRIPTION
Presentation from WebClusters 2012 conferenceTRANSCRIPT
Geoindexingwith
MongoDBLeszek Krupiński
WebClusters 2012
About me
On-line since 1997
Funny times
1 hr of internet for 1 USD
First social site:geocities
My first web page
What do I do now
Day-time jobManaging team of developers for Polish Air Force
Side:consulting, optimizing,
desiging
Buzzwords incoming!
The Internet2008
Web 2.0
http://en.wikipedia.org/wiki/File:Web_2.0_Map.svg CC-BY-SA-2.5
Be social in your bedroom
alone.
The Internet2012
Web 3.0
Why geospatial?
Needs shifted
Why?Because they could.
How to implement?
Database. Duh.
Keep, but also query
Is there a person at
53.438522,14.52198?
Nope.
Is there a person at
53.438522,14.52199?
Nope.
Is there a person at
53.438522,14.52199?
Yeah, here’s Johnny!
Not too useful.
Give me nearby homies.
Within the range of 1 km there is:
• Al Gore (53.438625,14.52103)• Bill Clinton (53.432531,14.55127)• Johnny Bravo
(53.438286,14.52363)
Now that’s better.
Geoindexing.Nothing new.
Oracle, PostreSQL, Lucene/Solr, even
MySQL (via extensions)
SELECT c.holding_company, c.location FROM competitor c, bank bWHERE b.site_id = 1604 AND SDO_WITHIN_DISTANCE(c.location, b.location, ’distance=2 unit=mile’) = ’TRUE’
ORACLE
SQL is so last year
Let’s use something cool
MongoDB.Because all the cool kids use NoSQL now
Why MongoDB?
Choose your NoSQL wise.
NoSQL in MongoDB
• Document –based• Queries (JS-like syntax)• JSON-like storage
Why MongoDB?Use Cases• Archiving• Event logging• Document and CMS• Gaming• High volume sites• Mobile• Operational datastore• Agile development• Real-time stats
Features• Ad hoc queries• Indexing• Replication• Load Balancing• File Storage• Aggregation• Server-side JavaScript• Capped collections
http://en.wikipedia.org/wiki/Mongodb
Back to geo.
{loc: [ 52.0, 21.0 ],name: ”Warsaw”,type: ”City”
}
db.nodes.ensureIndex({loc: '2d'})
That’s it.
Query• Exact
o db.places.find( { loc : [50,50] } )
• Nearo db.places.find( { loc : { $near : [50,50] } } )
• Limito db.places.find( { loc : { $near : [50,50] } } ).limit(20)
• Distanceo db.places.find(
{ loc : { $near : [50,50] , $maxDistance : 5 } }).limit(20)
Compound index• db.places.ensureIndex(
{ location : "2d" , category : 1 });
• db.places.find({
location : { $near : [50,50] },category : 'coffee‚
});
Bound queries• box = [
[40.73083, -73.99756],[40.741404, -73.988135]
]• db.places.find(
{"loc" : {"$within" : {"$box" : box }}
})
Problems
Units
Coordinates in arc units Distance in
kilometers
In query
earthRadius = 6378 // km
multi = earthRadius * PI / 180.0
range = 3000 // km
… maxDistance : range * multi…
In results
pointDistance =
distances[0].dis / multi
Earth is not flat.
Problem: can’t use linear distance
Earth isn’t flat too.
Solution?Use approximation.
MongoDB has it built-in
distances = db.runCommand({ geoNear : "points", near : [0, 0], spherical : true,
maxDistance : range / earthRadius /* to radians */} ).results
Focus: runCommanddistances = db.runCommand({ geoNear : "points" …
Sort by distanceOnly with runCommand
Automatically sorted• db.runCommand(
{ geoNear : "places" , near : [50,50], num : 10 });
• {"ns" : "test.places","results" : [
{ "dis" : 69.29646421910687, "obj" : … },{ "dis" : 69.29646421910687, "obj" : … },…
],…
}
Demo
OpenStreetMaps database of Poland
imported into MongoDB
14.411.552 nodes
3GB of raw XML data
PHP in virtual machine
Imported about 100.000 nodes every
10s.
Pretty cool, eh?
Kudos to Derick Rethans
Part of this talk was inspired by his talk
Questions?
Thanks!Rate me at https://joind.in/talk/view/6475
Geoindexingwith
MongoDBsupplement
Leszek KrupińskiWebClusters 2012
Why MongoDB?
Evaluate.
PostGIS is cool too.(but it’s SQL, meh)
Why MongoDB?Use Cases• Archiving• Event logging• Document and CMS• Gaming• High volume sites• Mobile• Operational datastore• Agile development• Real-time stats
Features• Ad hoc queries• Indexing• Replication• Load Balancing• File Storage• Aggregation• Server-side JavaScript• Capped collections
http://en.wikipedia.org/wiki/Mongodb
If you need other features of MongoDB,
use it
If you don’t, evaluate.
Evaluate.
Demo(hopefully)
Questions?
Please leave feedback!Rate me at https://joind.in/6475