Download - Condos in the BigData Knowledge Garden
Condos in the Big Data Knowledge Garden
Jack Park [email protected]
BigData Science Meetup Shyam Sarkar, Organizer
Fremont California, 12 May, 2012
© 2012, TopicQuests This work is Licensed under a Creative Commons Attribution-
NonCommercial-ShareAlike 3.0 Unported License.
A Personal Journey
Built airplane
Undergrad study: aero eng.
Built windmills/solar homes
Built wind/solar sensors
AI to analyze data
Collective intelligence
Knowledge gardens
It’s a waterfall life…
Towards a Big Data Rosetta Stone
• The term Rosetta stone has been used idiomatically to represent a crucial key to the process of decryption of encoded information, especially when a small but representative sample is recognized as the clue to understanding a larger whole.*
• Let us explore that idea with open source socio-technological infrastructures
http://en.wikipedia.org/wiki/File:Rosetta_Stone.JPG
*http://en.wikipedia.org/wiki/Rosetta_Stone
“Discovery consists of seeing what everybody has seen and thinking what nobody has thought.”
–Albert Szent-Györgyi
Big Data Gardening
• Apply ongoing organic (capable of generative self-organization) processes to large and continually growing datasets – Tended from the point of
view of individuals, of groups and topic-oriented projects, and of globally connected gardens in the jungle of knowledge.
http://en.wikipedia.org/wiki/File:The_Thinker,_Rodin.jpg
What is a Knowledge Garden?
Federated Information Resources
Condo (Topic
Garden)
Condo (Topic
Garden)
Condo (Topic
Garden)
Collections of participants at topic-centric portals (condos) with all information resources federated
A Knowledge Garden fuses Social with Technical and Political
What is a Condo
• A Social Contract – Sensemaking
• Collaboration • Collective Intelligence
– Discovery – Learning
• User Experience – Epistemic games – Structured conversations – Social gardening
• Tagging • Annotating • Connecting
Federation*
Conversation Tools
Topic Map
Research Tools
Other
Games
* http://www.knowledgefederation.org
Garden Workflow Big Picture
Encyclopedia
World of Information
Sources
•Collect and curate information resources •Transfer them to a Workbench
•Annotate, Tag, Connect ideas at the Workbench •Create summary topics in the Encyclopedia
•Blogs, Reports, Journalism, … •Organize Workbench resources in Topic Map
•Federate, share,…
Topic Map
Workbench
Topic Maps
• A Topic Map is like a library* – A Topic Map is indexical
• Like a card catalog – Each topic has its own representation
• Improving on a card catalog, a topic can be identified many different ways
– A Topic Map is relational • Like a good road map
– Topics are connected by associations – Topics point to their occurrences in the territory
– A Topic Map is organized • Multiple records on the same topic are co-located in the
map
*A library without all the books: a map is not its territory
From the perspective of complexity theory, emergence arises from complex systems that create new properties from “autonomous unities coming together into larger, more powerful unities”
Why Garden?
Olen Gunnlaugson (2011). A Complexity Perspective on Presencing. Complicity, Vol 9, No 1 (2012). Online at: http://ejournals.library.ualberta.ca/index.php/complicity/article/view/9624/8561
A Scenario—Federating Disparate Conversations and Discovery
Discovery?
Similar conversations can occur across disciplines with Big Data
Joe is in Medical School studying to be a physician
Joe is preparing for a test on Bacterial Infections
Scenario: Meet Joe
Immune Response
Bacterial Infection
Macrophage
Free Radical
Reactive Oxygen
macrophages use free radicals to
kill bacteria
Bookmark page with these tags
Annotate page with this idea Use garden condo tools to
record discoveries
Scenario: Joe’s Gardening
Web page discovered by Joe
Sarah lives a healthy lifestyle
Sarah reads that free radicals can cause cancer
Scenario: Meet Sarah
Antioxidant
Free Radical
Bookmark page with these tags
antioxidants kill
free radicals
Annotate page with this idea Use garden condo tools to
record discoveries
Scenario: Sarah’s Gardening
Web page discovered by Sarah
Ben is a cancer patient with recurring bacterial infections
Ben visits a garden condo to research bacterial infections
Scenario: Meet Ben
Ben makes a discovery Ben becomes curious about free radicals—a new concept to him
Scenario: Ben’s Garden Research
Free Radical
macrophages use free radicals to
kill bacteria
antioxidants kill
free radicals
The Free Radical tag has created a kind of wormhole (tunnel) that links ideas together. The opportunity is to recognize the importance of those connections.
antioxidants kill
free radicals
macrophages use free radicals to
kill bacteria
Ben adds a new Connection between
the two ideas
Must Avoid
Scenario: Ben’s Gardening
Ben has an epiphany Sounds like I should not take any more antioxidant pills!
Emergence: In philosophy, systems theory, science, and art, emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions.*
*http://en.wikipedia.org/wiki/Emergence
Olivia is a Medical Doctor interested in immune system subjects
Scenario: Meet Olivia
antioxidants kill
free radicals
macrophages use free radicals to
kill bacteria
Need to restate this information
“Must Avoid” doesn’t really
convey a proper sense of reality
Must Avoid
Scenario: Olivia’s Garden Research
Olivia visits the garden
antioxidants kill
free radicals
Contraindicates
macrophages use free radicals to
kill bacteria
Bacterial Infection Antioxidants
Because
Create a new set of representations by connecting the two core concepts and
explaining the connection
Olivia gets bonus points for justifying her assertion.
Use garden condo tools to record discoveries
Scenario: Olivia’s Gardening
Contraindicates Bacterial Infection Antioxidants
Scenario: Starting a Conversation
A Challenge is posed to Olivia’s primary
assertion
“Compromised host” as a new concept to be represented in the
garden’s topic map
Use garden condo tools to record discoveries
Scenario: The Conversation Structured conversation responses to the question
antioxidants kill
free radicals
Contraindicates
macrophages use free radicals to
kill bacteria
Bacterial Infection Antioxidants
Because
Appropriate For
Compromised Host
Use garden condo tools to record discoveries
Scenario: Garden Results Co-created resource in the garden
Big Data
Use Cases for a Big Data Garden
• Resource Collection – Federation
• bring together and organize without filters
• Resource Augmentation – Tagging – Annotating – Debate
• Knowledge Cartography – Connecting resources – Map maintenance – More Debate
• Research Augmentation – Crowd-sourced discovery – Automated inferences /reasoning – Knowledge sharing
Federated Information Resources
Condo (Topic
Garden)
A View of Big Data Computing
• Solr + Hadoop = Big Data Love*
– Article gives a hint at an approach to using off-the-shelf open source software to support Big Data
– We consider it as a baseline for design
*(concept and image)http://architects.dzone.com/articles/solr-hadoop-big-data-love
Candidate Federation Architecture
Know New Condo New Condo
New Condo New Condo Other Topic
Gardens
New Condo New Condo
New Condo New Condo
Political Topic Garden
TopicQuests Garden Topology
IdentiMe
Personal Microblog
TopicQuests Non-Profit
Read Only Bulk User
TopicQuests Non-Profit
API
KG Admin
Free Access User
API
New Condo New Condo
New Condo New Condo Other Topic
Gardens
New Condo New Condo
New Condo New Condo
Medical Topic Garden
New Condo New Condo
New Condo New Condo Other Topic
Gardens
New Condo New Condo
New Condo New Condo
Education Topic Garden
New Condo New Condo
New Condo New Condo Other Topic
Gardens
New Condo New Condo
New Condo New Condo
Other Topic Garden
Support / R&D
Add-On Agents
Knowledge Garden
Knowledge Federation
Server
A Strong Conjecture
• A Knowledge Federation’s topic map provides a Rosetta Stone substrate
– Supports reasoning by analogy
– Big Data population studies (probabilities and statistics) provide clues added to the map
– A well-organized map of the growing territory provides a navigable terrain to support journeys of discovery
Completed Representation
antioxidants kill
free radicals
Contraindicates
macrophages use free radicals to
kill bacteria
Bacterial Infection Antioxidants
Because
Appropriate For
Compromised Host
Co-create Gardens and Federate Big Data Science
Thanks to Mark Szpakowski , Patrick Durusau , and Martin Radley for valuable comments