open data, visualization & usability for online news delivery
DESCRIPTION
A presentation by Mohanaraj Gopala Krishnan and Goh Sze Ying at the Asia New Media Forum “Next Generation News” Kuala Lumpur, June 11th - 12th 2008TRANSCRIPT
1
Open Data, Visualization &Usability for Online News Delivery a presentation by Mohan & Sze
2
Background
3
• Lack of structured access to information on local news sites
• POV of two technologists whose interests in technology and current issues/activism overlap
• Propagation of more aware and savvy communities
Who are we?
4
Questions
5
• know HTML and/or CSS?
• know Javascript?
• know server side programming?
• maintain a news site / website?
• monetize your content?
How many of you…
6
Open data
Visualisation
7
Your data wants to be free
8
Why?
You cannot define how people will consume your information, leverage them
• Don’t turn your user base into pirates
Why?
Monetization cannot solely be a result of exclusivity but a result of mass access and authority
• The more people use your data, the more valuable you become (advertising)
• How are you making your data (not just website) more accessible
Why?
The NY Times said the project had met expectations, drawing 227,000 paying subscribers — out of 787,000 over all — and generating about $10 million a year in revenue.
“But our projections for growth on that paid subscriber base were low, compared to the growth of online advertising...”
“...what wasn’t anticipated was the explosion in
how much of our traffic would be generated by Google, by Yahoo and some others.” Vivian L. Schiller, senior VP and GM of NYTimes.com
If you ain’t the top dog, use openness as a way to partner, grow market share and relevance
• Even Google does it - OpenSocial
• Leverage exiting platforms - don’t roll another SNS
Why?
No matter how smart you are, there are smarter people out there
• Do you want them to be your competitor or partner
• Who will be making your killer Facebook application ?
Why ?
17
How?
How
• Provide a way to access data without presentation clutter. Turn your sites into information services
• Make it structured & machine readable - Web APIs, Microformats
How
• Provide simple ways to export and import data into your system
• Syndication, public wikis
• Provide ways to simplify the querying of your indexed data
• Opensearch, Better back end search
How
21
Microformats
• Small bits of HTML that represent things like people, events, tags, etc. in web pages. (annotations)
• These small bit of standardized markup adds semantic meaning to data in a page in a non-intrusive and machine readable manner, data becomes more structured
• A bottom up approach - community defines what and how the markup will be
Microformats
23
Let’s jump right in!
<table class="contentpaneopen"> <tbody> <tr> <td colspan="2" class="createdate" valign="top">Wednesday, 28 May 2008 </tr> <tr> <td colspan="2" valign="top"> <p><img src="forum_location_files/pjmap.jpg" alt="Image" title="Image" border="0" height="367" hspace="6" width="497"></p> <p>Shah Village Hotel<br>No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia </p> <p>Hotel Information: <a href="http://www.hspj.com.my/HSGS/index.php">Singgahsana Hotel</a>, <a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a></p> </td> </tr> <tr> <td colspan="2" class="modifydate" align="left">Last Updated ( Friday, 06 June 2008 )</td> </tr> </tbody></table>
<table class="contentpaneopen"> <tbody> <tr> <td colspan="2" class="createdate" valign="top">Wednesday, 28 May 2008 </tr> <tr> <td colspan="2" valign="top"> <p><img src="forum_location_files/pjmap.jpg" alt="Image" title="Image" border="0" height="367" hspace="6" width="497"></p> <p>Shah Village Hotel<br>No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia </p> <p>Hotel Information: <a href="http://www.hspj.com.my/HSGS/index.php">Singgahsana Hotel</a>, <a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a></p> </td> </tr> <tr> <td colspan="2" class="modifydate" align="left">Last Updated ( Friday, 06 June 2008 )</td> </tr> </tbody></table>
<div class=“vcard”>
Shah Village Hotel<br>No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class="vcard">
Shah Village Hotel
No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class=“vcard”>
Shah Village Hotel<br>No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class="vcard">
<div class="fn org"> Shah Village Hotel</div>
No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class=“vcard”>
Shah Village Hotel<br>No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class="vcard">
<div class="fn org">Shah Village Hotel</div>
<div class="adr"> No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class=“vcard”>
Shah Village Hotel<br>No. 3 & 5 Lorong Sultan , 46200 Petaling Jaya, Selangor D.E. Malaysia
</div>
<a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class="vcard"> <div class="fn org">Shah Village Hotel</div> <div class="adr"> <div class="street-address"> No. 3 & 5 Lorong Sultan </div>,<span class="postal-code"> 46200 </span> <span class="locality"> Petaling Jaya </span>, <span class="region"> Selangor D.E. </span> <span class="country-name"> Malaysia </span> </div> </div><a href="http://www.shahsresorts.com/petalingjaya/default.htm">Shah's Village Hotel</a>
<div class=“vcard”> <a href= "http://www.shahsresorts.com/petalingjaya/default.htm" class=“url fn org”>Shah Village Hotel</a> <div class="adr"> <div class="street-address"> No. 3 & 5 Lorong Sultan </div>, <span class="postal-code"> 46200 </span> <span class="locality"> Petaling Jaya </span> , <span class="region"> Selangor D.E. </span> <span class="country-name"> Malaysia </span> </div> </div>
<div class=“vcard”><abbr title="3.102233488513198;101.64600670337677"class="geo"><a href= "http://www.shahsresorts.com/petalingjaya/default.htm" class=“url fn org”>Shah Village Hotel</a> </abbr> <div class="adr"> <div class="street-address"> No. 3 & 5 Lorong Sultan </div>, <span class="postal-code"> 46200 </span> <span class="locality"> Petaling Jaya </span> , <span class="region"> Selangor D.E. </span> <span class="country-name"> Malaysia </span> </div> </div>
32
Second example
• Search engine support
• http://kitchen.technorati.com/search/
• google creative commons search
• yahoo search support
Why?
34
• Machine readable
• Cheap API - Low barrier for community to build add ons to your site tools in (Javascript - FF3 , PHP etc.)
Why?
35
• End user application support (Firefox3 with extension,IE8 OpenService - WebSlices , soon more apps)
Integrate into news sites:
•Listings of classifieds - hListings
•News stories - markup with hAtom, markup authors,personalities and organizations with hCard, markup locations with geo and adr
•Reviews of products,restaurants, movies - hReview
e.g. http://www.australianit.news.com.au/contactus
How to incorporate
How to incorporate
Build tools internally that leverage the markup
•Internal search engine that is markup aware for your site
•Build visualisation tools/widgets that leverage the markup
How to incorporate
Build communities that leverage the markup
•Have a developer section on your site
• DreamweaverExtension - http://www.webstandards.org/action/dwtf/microformats
• Textpattern Microformats Plugin (http://placenamehere.com/TXP/pnh_mf/)
• Wordpress Microformat plugins - http://microformats.org/wiki/wordpress
• TYPO3 - tt_address , TIMTAB Weblog plugins
• Joomla - No support
• Drupal - Drupal Upcoming.org syndication module-http://hybernaut.com/upcoming-hcal - Weak support
• PHP Lib - http://enarion.net/phpmicroformats/ (Note: Search for otherlanguage libs to generate microfrmats)
Produce
• Client end
• Microformat bookmarklet - http://leftlogic.com/lounge/articles/microformats_bookmarklet
• Firefox add on - Operator-http://www.kaply.com/weblog/operator/
• Server end
• Python based microformat parser - http://code.google.com/p/aump/
• PHP based microformat parser - http://code.google.com/p/hkit/
• TransFormr -A Simple set of XSLT and PHP tools for Transforming
• Microformats - http://code.google.com/p/transformr/
• http://suda.co.uk/projects/x2v - webservices
Consume:
• Weak tool-ing for production in open source components such as Joomlaand Drupal
Challenges:
SyndicationAccess of content + metadata through feeds. A data API for your site.
Users will consume your data in the way they find most comfortable. Syndication allows developers/users to build the right method for themselves. (Mobile readers, Screen readers). Bottom line - increase in circulation.
Most Malaysian news sites do it badly
Why?
Why?
Additional metadata contained within a feedallows for remixing and discovery. Categories and tags (metadata) should be taken seriously. (Technorati, wordpress,yahoo pipes )
Why?
Allows for 3rd party applets/widgets to be built on top of your content
• Integrate into news sites:
• Syndicate everything on your site
• All sections , all columns ,search results, listings (classifieds etc.)
How:
• Drupal Atom module - http://drupal.org/project/atom - Overall good support
• Joomla - DS-Syndicate-http://extensions.joomla.org/component/option,com_mtree/task,viewlink/link_id,2497/Itemid,35/(Only Joomla 1.0 though) - Bad support
• Wordpress - Good support
• Universal Feed Generator -http://www.phpclasses.org/browse/package/4427.html - Roll your own
Produce
•Simplepie PHP library - http://simplepie.org/development/
•Drupal - FeedAPI - http://drupal.org/project/feedapi
Consume
•Use Atom 1.0
•Cache, cache, cache (Support conditional GETs, Slugs etc.)
•Autodiscover links, on all pages - leverage support in browsers
•Use service to redistribute - such as Feedburner
Best practice:
Monetization:
If people aren't returning to my site for content, how do I monetize ?
Challenges:
A collection of simple formats that help searchengines share their search results.
3.0 OpenSearch
Browser support is already available
• Firefox, IE7 onwards.
Why?
Why?
Leverage search aggregation, deep search
• Growing field - A9, Technorati, MS Search Server 2008 OpenSearch
Why?
Weak search access results in searching through Google or Yahoo?
• Your customers are looking for your data on other sites - hence you are loosing usage metrics as well as monetization opportunities.
• Integrate into news sites:
• Make search a 1st class interface to your website.
• Why are you not using this browser UI element ?
How
62
63
• Ensure everything is indexed and can be searched
• Elevate search from an afterthought into a first class access tool of your site
How
• Textmate - wet_opensearch plugin
• Drupal - OpenSearch Plugin - - http://drupal.org/project/opensearchplugin
• Online creator tool - http://ready.to/search/en/
• Wordpress - plugin - http://wordpress.org/extend/plugins/open-search-document/
Produce
Drupal - OpenSearch client - http://drupal.org/project/opensearchclient
Consume
1. Use autodiscovery links
2. Write informative OpenSearch description documents
3. Provide atom + microformat + extensions (geo) based search resultswhere applicable
Best practice
67
Open data
Visualisation
68
Bad design obscures.
Good design reveals.
69
• Design is about constraints – understanding, respecting and breaking.
• An effective design allows users to get what they’re looking for in an easy, trusted and fast manner.
So, what’s the deal with design?
70
Traditional vs social mediaHow are they different?
71
72
73
1. Dynamic content (static vs changeable)
2. Interactivity (comments)
3. Relevance / Popularity (unique visits, comments, linkbacks)
4. Accessibility (archives)
5. Mashup (mix media, mix services – ads, widgets)
6. Creative freedom
7. Continuous presence
8. Reusable (syndicatable, linkable, tagable)
74
Social media = interactions between people using
applications / interfaces
75
• improve the interactions between audience
• prioritize information – hierarchical structure – popularity, relevance, recency et. al.
• allow users to reorganise and remix content and represent content in a manner they desire.
Why?
76
SIMILE Timelinea Javascript-based widget for visualizing time-based information
77
• linear narrative – visual representation of chronological events
• allows users to be able to appreciate and interact with news in a more fluid, continuous form – better context to events & stories
• Mashup – information can be easily extracted from different sources (e.g. http://www.bbc.co.uk/history/british/launch_tl_british.shtml )
Why?
78
• http://simile.mit.edu/timeline/
• Timeline is similar to Google Maps: zero application installation, server-side/client-side
• Data is easily populated into Timeline using: RDF, RSS, JSON, Atom, XML
How
79
<html>
<head>
...
<script src="http://simile.mit.edu/timeline/api/timeline-api.js“ type="text/javascript"> </script>
...
</head>
<body>
...
</body>
</html>
80
<html>
<head>
...
<script src="http://simile.mit.edu/timeline/api/timeline-api.js“ type="text/javascript"> </script>
...
</head>
<body>
...
</body>
</html>
81
82
Timestampinga Javascript-based widget for visualizing time-based information
83
• Video markers create easy transitions for readers to move between different paragraphs of a news article and its corresponding video clip
• Limited time - short attention span – users has the freedom to view specific portions of a clip
Why?
84
• Interactive markers – viddler.com allows users to
1. attach comment markers at any point of the video clip;
2. include tags in the video timeline to describe elements in the video clip
How:
85
86
87
88
89
Google MapsBuilding humane interfaces by respecting standards and
habituation
90
• Topographical view of how news is scattered allows for readers to focus on hyper local content
• Maps also serve as infographics to augment readers’ experience in relating or understanding news, especially if there is a location specific context e.g. Cyclone Nargis (http://www.nytimes.com/interactive/2008/05/08/world/20080508_MYANMAR.html)
Why?
91
• zero application installation, server-side/client-side
• data is easily populated with XML
How:
92
• Human readable also machine readable
• Better manipulate the data - move towards machines autonomously aggregating, remixing and manipulating data
• Make your data accessible - make yourself important
• Monetize by selling attention -but don’t lose relevance
• Data + Visualization = More attention
Summary