apache solr changes the way you build sites
DESCRIPTION
By Jacob Singh and Peter Wolanin about the Apache Solr Search Integration module for Drupal. Presented at Drupalcon Paris, 2009.TRANSCRIPT
APACHE SOLR CHANGES THE WAY YOU BUILD SITES
How to build dynamic navigation for dynamic content
Jacob Singh and Peter Wolanin Drupalcon Paris, September 3rd 2009
RELEVANT AND FAST
FILTERS
SORTABLE
MORE LIKE THIS
SITE NAVIGATION
INFORMATION ARCHITECTURE
•Is the science and art of guessing what your users want to see or do on your site and helping them get there
•Often done without actually consulting visitors or proper understanding of the target market
ARCHETYPES - WHO
CONTENT - YOU GOT IT
ORGANIZE AND PROSPER!
NAVIGATION
SO? WHAT’S WRONG WITH THAT?
Site Map!
(1996)
WEBSITES GROW.
•Sometimes really fast.
• And always in ways you didn’t expect
WHY DOES THIS HAPPEN?
VISITORS ARE TOO VARIED TO THINK OF THEM ALL
YOUR CONTENT IS TOO VARIED FOR YOUR MENU
WHERE ARE BEER HATS?
YOUR CONTENT IS TOO VAST AND TAKES TOO MANY
CLICKS TO FIND
TOOK 7 CLICKS TO FIND DRUPAL
YOUR CONTENT IS CONTRIBUTED BY USERS
AND UNPREDICTABLE.
DRUPAL IS A SOCIAL PUBLISHING PLATFORM
YOU USE IT SO USERS WILL ADD CONTENT
That’s the point!
IF WE HAVE DYNAMIC USER CONTRIBUTED CONTENT
WHY DO WE INSIST ON STATIC ADMIN DEFINED
NAVIGATION?
IF WE HAVE DYNAMIC USER CONTRIBUTED CONTENT
WEB 2.0 JARGON TO SAVE THE DAY
•Tag Clouds
•Content Recommendation
•Social Networking
•Social Bookmarking
HOW SEARCH DIED AND
HOW WE BRING IT BACK
! ?
SEARCH IS NOT A PRIORITY
MOST DRUPAL WEBSITES REALLY IGNORE SEARCH
CAN YOU FIND THE SEARCH BOX?
HINT: I’VE CIRCLED IT IN ORANGE
WHAT DO THE BIG SITES DO?
WHY DID SEARCH DIE?
•It was too slow
•It wasn’t smart enough
•Users learned not to trust it
LANGUAGE IS IMPORTANT
GOLDEN RULE: No Dead Ends!
SEARCH ON G.D.O
FIND OUT WHERE TO LOOK
FOUND ‘EM
I LEFT MY TIME MACHINE AT HOME
SMARTER MATCHES
STEMMING
SPELLING SUGGESTIONS
BUT I’M THE ADMIN!
ALL THE COOL KIDS ARE DOING IT
DAMZ EN L’MASSION
A TAILOR MADE NAVIGATION FOR EVERY USER
You want it, right?
The Apache Solr Project
• Stable and proven.– Used by Netflix, CNET, CitySearch, StubHub!, GameSpot, AOL– Full time maintainers– VERY Active mailing list (about 1k messages per month)
• Fast: written in Java.• Uses Lucene: the top open source search
library.• Distibuted: scales out in multiple directions.
Apache Solr Search Integration
• Very active project on drupal.org.• Takes advantage of latest Solr features.• Exposes an API to modify search and display
behavior.• Supported by engineers at Acquia.• All Acquia code improvements have been
contributed back to the Drupal.org project.• Many of the Drupalcon sponsors and attendees
are already involved and using it.
Feaure highlights
• Taxonomy, user, and language facets.• Node type faceting, weighting, and exclusion.• Node property (e.g. sticky) and date weighting.• Date facets on content creation or change.• OG facets (optional sub-module).• Node access respected (optional sub-module).• More-like-this content recommendations.• Customizable (see drupal.org module browsing
features).
Integration with other modules/data
• Webmail: http://drupal.org/project/webmail_plus
• Attachments: http://drupal.org/project/apachesolr_attachments
• Views: http://drupal.org/project/apachesolr_views
• Services: http://drupal.org/project/solr_service
• Nodequeue: http://drupal.org/project/nodequeue
• RDF: http://drupal.org/project/apachesolr_rdf
• Project: http://drupal.org/project/project
• Ubercart: http://drupal.org/project/apachesolr_ubercart
You Can Run Yourself. Easy!1. Get a dedicated server or a VPS and get Solr loaded on it.
2. Find a Java server administrator or get some books.
3. Get the Drupal module, install the PHP library, and configure it.
4. Replace the stock Solr configuration files with Drupal ones.
5. Learn about Solr replication and configuring it.
6. Set up log management, alerting, monitoring, etc.
7. Implement regular upgrades or patches to Solr which will requiring getting your Java development set up and building from source sometimes.
8. Keep up to date with the Drupal module.
9. Implement a security regime to protect data transfer (i.e. so spammers can’t add Viagra ads to your search results)
10. Harden your servers, setup firewalls and IP-based, password-based, or other security.
11. Figure out how handle updates and versioning of Solr and your schema.
12. *Recommended: Get on the solr-user and solr-developer mailing lists to get updates and alerts on the Apache Solr project. Don’t worry, it’s only a 50 or so mails a day if you don’t count the commit messages.
Or... use Acquia Search
• Sign up on acquia.com.• Free 30 day trial subscriptions for anyone.• You must be running a Drupal 6.x site, with PHP
5.2.0+ (5.1.4+ possible as well).• Use Acquia Drupal or install our search module
package.• It leaves Drupal core search intact, so you can
go back anytime.• Convert your site and start impressing users!• We will worry about everything else.
How Acquia Search Works
Your webserver
Search slave servers
Search master server
AcquiaNetwork
content to index
SSL, HMAC
authenticatedrequest
indexreplication
search request
SSL, HMACauthenticated
results
SSL, HMAC
Proving the platform
• Benchmarking our servers, on the search server itself, most searches run in < 200 ms, even under high load.
Who Is This For?
• Small and medium size sites - easy access to enterprise search for every Drupal site.– No hardware, no experience, fast setup, low cost.
• Large sites and Acquia partners - the same solution you’d deploy, but faster and easier.– Don’t consume your engineering resources.– Why load your own servers? – We handle the security and availability.– Impress your users and clients.
How-to Screencasts
• http://www.jeffnoyes.com/content/how-use-acquia-search-apache-solr
• http://www.jeffnoyes.com/content/enabling-acquia-search-and-apache-solr
Here’s a quick look at the admin interface:
© 2008 Acquia, Inc.
New modules to enhance the experience
© 2008 Acquia, Inc.
Searching with Views 3• Define filters, sorts in Views like normal• Solr, not MySQL, gets the query• Views is responsible for rendering
results:• you configure the visible fields• grid views• carousels • search results in blocks
• Faceting works like in current Apache
© 2008 Acquia, Inc.
Javascript interface• http://drupal.org/project/solrjs
© 2008 Acquia, Inc.
Javascript interface• http://drupal.org/project/solrjs
© 2008 Acquia, Inc.
Autocompletehttp://drupal.org/project/apachesolr_autocomplete
© 2008 Acquia, Inc.
Stats for administratorshttp://drupal.org/project/apachesolr_stats
© 2008 Acquia, Inc.
DRUPAL-6--2 Branch
© 2008 Acquia, Inc.
DRUPAL-6--2 Branch
• Researching more substantial architectural changes to query building.
• Looking at ways to support multi-site and better support multi-lingual content.
Learn and Contribute• Find us at the code sprint (Saturday) if you have
questions about the code or roadmap.• Come to the Acquia table any time to learn
more about Acquia Search.
[email protected]@acquia.com
Thank you!
© 2009, Acquia, Inc.