Download - WorldOS Corp
WorldOS Corp.WorldOS Corp.
NYJavaSig Talk 1/30/2001
IntroductionIntroduction
Me– Lucas Gonze– Web/eCommerce developer– Open source culture– Network emphasis
IntroductionIntroduction
WorldOS– One year old– Topology centric– Java– About the software– About the business
IntroductionIntroduction
Web era led to Java and Windows standoff
In general Internet era, what next?– .Net & C#?– Java on the client?– Client->Server or Server->Client?
• Whither Perl, Python and VB?
OverviewOverview
This talk will– Introduce peer to peer ideas– Evaluate Java as a tool for peer to peer.– Show common approaches– Show infrastructure you can use
Focus on practical knowledge over theory.
Basics of Peer to PeerBasics of Peer to Peer
Working definition of P2P: Projects that followed directly on Napster.
P2P networks often allow computers to act as both client and server.
Every computer connected to the network will probably provide resources to the system as a whole. That is not as true for a web browser.
Basics of Peer to PeerBasics of Peer to Peer
You only own your side of the connection. The other side is a black box.– It is not safe to assume that you wrote the software on the
other side. You have no way of knowing what is on the other end of a connection.
– This precludes approaches like storing unsigned state in outgoing messages, and assuming that it is canonical when it comes back.
– Use nodewise encapsulation to leverage this heuristic.
Connections between nodes are interfaces, not buses. Design for rough grained transactions.
Client/Server Job SearchClient/Server Job Search
jobboard1.com
jobboard3.com
jobboard2.com
employer
employer
job seeker
job seeker
job seeker
Interconnections must be mediated
P2P Job SearchP2P Job Search
jobboard1.com
jobboard3.com
jobboard2.com
employer
employer
job seeker
job seeker
job seeker
1 : request
2 : forward
3 : forward
2 : forward
3 : forward
4 : match
Interconnections may be unmediated
Families of P2P applicationsFamilies of P2P applications
Put “dark (but normal) matter” to work– CPU
• Seti@home, PopularPower
– Files• Napster, Gnutella
– Bandwidth• Swarmcast
Families of P2P applicationsFamilies of P2P applications
Use the unique properties of decentralized computer networks– Privacy
• Publius
• Freenet
– Complex systems • MojoNation
• Random number generator
• Economic modeling
• Cellular automata
An Example: SwarmcastAn Example: Swarmcast
Parallel streams via rebroadcast Immunizes against /. effect. As a piece of
content becomes more popular, downloads become faster to the consumer, and less costly to the provider.
Flood strategy. You can’t be inefficient in using stuff that would have been thrown out anyway.
HeterogeneousnessHeterogeneousness
P2P puts network effects to workPutting network effects to work means
large networksIn a very large network there is no boss
to enforce standardsTo thrive without standards:
– Use LCD approach– Be adaptable
HeterogeneousnessHeterogeneousness
Don’t try to force it – the network is bigger than you.
Jujitsu is the only chance you have.Think of the Fed manipulating the US
economy via the prime rate, rather than by wage and price controls.
Heterogeneousness & JavaHeterogeneousness & Java
Evaluating the suitability of Java to P2P is a question of how well Java helps you to exploit network effects.
A Java program is often poorly adapted to local conditions like “users are not computer experts”. A native program can be optimized for a dominant platform. This allows it to
– target and take advantage of local conditions (e.g. with a newbie oriented GUI), and
– leverage the ensuing popularity to drive native implementations on other platforms.
– Such a program thrives by avoiding heterogenous situations. Not a bad strategy!
So how does Java stack up in taking advantage of network effects? That is, if you are are trying to use jujitsu to get network effects on your side, how does Java measure up the problem?
Heterogeneousness & JavaHeterogeneousness & Java
Strategies: Avoid heterogeneousness
– Example: Microsoft applications software works extremely well, but only in Windows
Stick to least common denominators– Example: Email works everywhere, for everyone, but only if
we all stick to the standards at the time email became popular - ASCII and SMTP.
Be as adaptable as possible– Example: A P2P project with an IE plugin that allowed
searching Gnutellanet, Napster and Google at the same time.
Java’s ability to exploit LCDsJava’s ability to exploit LCDs
HTTP– Pro: Jakarta-Tomcat works great– Con: Tomcat requires Java 2 or better
ASCII– Pro: Java’s support for ASCII is second to none– Con: …
Java is itself designed to be an LCD– Pro: JVMs < Java 2 are common on user machines– Con: JVMs >= Java 2 are rare on user machines
Windows– Pro: popularity of Mac and Unix.– Con: ubiquitousness of Windows.
Java’s ability to exploit LCDsJava’s ability to exploit LCDs
Conclusion: If Java 2 isn’t needed Or if your users will download the JRE(*) Or if your app needs to support Mac and Unix
usersThen Java is a good bet for exploiting LCDs.
*broadband makes this reasonably likely for high end users; dialup makes this unlikely for consumer users.
Java’s ability to be adaptableJava’s ability to be adaptable
Multiprotocol– Pro: Java’s roots among network oriented programmers (Unix people)
make it the killer environment for TCP development.• There exist excellent implementations of
– SMTP (via Javamail)– NNTP– HTTP– Gnutella– CORBA– RMI– SOAP/HTTP
– Con: No support for raw IP. • No implementation of “Ping”!
Why does multiprotocol matter? Network effects of joining multiple subnets.
Java’s ability to be adaptableJava’s ability to be adaptable
Hardware agnostic – handles multiple binary formats gracefully– Pro: Network-order byte stream everywhere
– Con: C has better tools for binary operations.
– Con: RMI is Java-to-Java only. WORA
– Pro: not bad, all things considering. No serious competition in this area, because no other interpreted languages have the same range of support.
– Con: not nearly good enough.
Java’s ability to be adaptableJava’s ability to be adaptable
Flexible trust models– Pro: Huge range of encryption, AI and other modern
tools.– Pro: Fine grained sandboxing policies.– Pro: Runtime loading and evaluation allow great
flexibility.– Pro: Java’s user base in the complex systems academic
communities– Con: CPU intensive work plays to Java’s weakness
(performance).– Con: MojoNation is the most mature toolkit, and that is
in Python. WorldOS is in Java, but is far from mature.
Java’s ability to be adaptableJava’s ability to be adaptable
XML – Pro: Allows for decentralized extension of message formats.
– Pro: Is unequalled cross-platform glue.
– Con: Has no efficient binary equivalent for high-volume messaging (yet).
Java XML support
– Pro: Industry standard XML tools are Java – Xerces, Xalan, JDOM.
– Con: Java 2 required for Xerces, Xalan, JDOM.
– Con: Parsers are slow.
Java’s ability to be adaptableJava’s ability to be adaptable
Conclusion:
If high adaptability is needed, Java is a good tool.
Real World ComparisonsReal World Comparisons
Gnutella– not Java: Bearshare
• Footprint 1.09 MB• Appearance pretty great
• Usability excellent throughout– (but partially due to much effort)
– Java: LimeWire• Footprint 6.39 MB
• Appearance somewhat ugly
• Usability relatively bad – (but not due to not trying)
– Does not match other native software
• e.g. the file open and save dialogs are not the standard
• E.g. you can’t sort a file list by clicking on sub-heads
– Buggy
• E.g. the “details” option in the file browse dialog doesn’t work
Real World ComparisonsReal World ComparisonsGnutella/Windows: BearShare
Real World ComparisonsReal World ComparisonsGnutella/Java: LimeWire
Real World ComparisonsReal World Comparisons
Unique features of Limewire– Community -- Enter an open community of similar users, as
defined by interests and computing power / bandwidth, to optimize search performance
– Anti-Freeloader Features - Choose to protect and enhance the gnutella network by "preferring" nodes who share files and information.
– Browse Host -- See all files a given host has – Library Function -- Launch and manage downloaded and
shared files – Relevancy Scoring -- See which search results are most
relevant to the search terms
ImplementationImplementation
Q: How do you get started?A: Look for precedents.
Implementation: Client+ServerImplementation: Client+Server
Simplest approach to converting a web app to a P2P app is to run the web server on localhost.– Minimal modifications to the code, if
you’re lucky.– Need a database, tag replacer, perl etc.
on client side.– Immature technology.
Implementation: Implementation: Desktop Web ServerDesktop Web Server A web server running on the client Webby UI
– Either • SOAP calls between native GUI and your web server.
– Interface can be written in VB
– Magi uses this approach
– OR– HTTP calls between browser UI and your web server
– With reasonable latency web UIs are much more usable.
Implementation: Implementation: Desktop Web ServerDesktop Web ServerDesktop web servers
– Pro:• HTTP and HTML are mature designs.• Can use web skills and tools.• With home and UI uncoupled, you have
remote access to the app.• Lower fixed costs to pay for centralized
servers.
Implementation: Implementation: Desktop Web ServerDesktop Web Server Desktop web servers
– Con:• You have to make a web server look and feel like a GUI
application.• Immature technology – you have to roll your own.• Security – how to avoid opening up the user’s machine to
crackers?• You still have to put up with web problems like browser
incompatibility and statelessness.• OS may not be well adapted
– Scheduling problems– DNS lookup of localhost when offline
• If everyone uses this approach then server sockets will be crowded.
Implementation: Implementation: Desktop Web ServerDesktop Web Server
– Java web servers:• Sun Brazil
– small and adaptable
• Apache Jakarta-Tomcat– Reliable, large, Java 2
• Apache Jserve– Reliable, large, Java 1
• Roll your own– Best fit, time consuming, you know the code
Implementation: Implementation: PublishingPublishing Like Freenet, MojoNation, FreeHaven Allows for remote disk storage. Splits data up across a number of different
computers. Heavy encryption and security demands. Use Freenet for a Java base See Uprizer for a commercial
implementation and professional services.
Implementation: Implementation: Data SynchronizationData SynchronizationAllows for synchronizing data across
multiple devices.For example, sharing your schedule
across your laptop, PDA, PC and cell phone.
To avoid reinventing the wheel WRT version control, use CVS or WebDAV
Use Magi for an Java base
Implementation: Implementation: InfrastructureInfrastructureOpen source Java projects that you
can work from:Publishing Freenet
Bandwidth Sharing OpenCola->Swarmcast
Decentralized FileSharing Furi
Hub and Spoke File Sharing Jnap (http://www.perham.net/mike/jnap/)
Desktop Web Server Tomcat, Jserve, Brazil
Security WorldOS
Adaptability Brazil, WorldOS
Data Synchronization Magi
SummarySummary
Use – Nodewise encapsulation– Rough grained transactions– Either LCDs or Adaptability– Java 2 for broadband users, Java < 2 if
your users are dialup.Take advantage of network effectsUse jujitsu
WorldOS Application EngineWorldOS Application Engine
The barebones skeleton of a peer node. A data interchange Plugin points for
– Protocol handlers
– “Peer CGI” (P-CGI) scripts
System ArchitectureSystem Architecture
Protocol MorphingProtocol Morphing
A protocol agnostic stance for transport and message protocols. This allows developers to use any collection of protocol that best fits their needs, enabling Goa users access to take advantage of resources on other networks.
Hence, a Goa user searching for a file could simultaneously query Napster, MojoNation, and Lightshare via a single user interface.
Protocol MorphingProtocol Morphing
Protocol morphing leverages the externalities of multiple networks, creating a ‘supernetwork’ for Goa users.
P2P applications
not to scale
MojoNation
Lightshare
NapsterWorldOS Goa
P-CGI Application InterfaceP-CGI Application Interface
Traditional Client/ServerTraditional Client/Server
Information and processing are distributed on the Internet, but much of the control [applications] is still greatly centralized.
internet
client
client
client
server
client/server network
P2P Peer NetworkP2P Peer Network
Peer technology blurs the distinguishing line between clients and servers, creating a network of peers, such that each node can behave as both client and server.
internet
peer
peer networkpeer
peer
peer
peer
Peer to Peer (P2P)Peer to Peer (P2P)
P2P networks allow computers to act as both client and server.
Each computer connected to the network provides resources to the system as a whole.
P2P is particularly useful for scalability and redundancy. Use of P2P to reduce single points of failure and controller bottlenecks is likely to be effective.
Data centers and Akamai-type mirrors are natural candidates for P2P.
The Status Quo Job SearchThe Status Quo Job Search
Client-Server Technology
A job seeker has to log onto multiple job sites, post resumes and search for jobs.
An employer must go through the same steps, seeking out relevant resumes and posting jobs.
The Status Quo Job SearchThe Status Quo Job Search
jobboard1.com
jobboard3.com
jobboard2.com
employer
employer
job seeker
job seeker
job seeker
internet job searching in the status quo
The Ideal Job SearchThe Ideal Job Search
The Ideal – Peer-Peer TechnologyA single request can be sent from a peer to its neighbors, being forwarded [via viral distribution] along to others. Hence, the success rate for a prospective employer or employee has a very high probability of success with minimal effort.
The Ideal Job SearchThe Ideal Job Search
employer
employer
job seeker
job seeker
job seeker
internet job searching in a peer network
job seeker
The Real Job SearchThe Real Job Search
Leveraging The Strengths Of Each System
This network is interoperable with currently installed systems and utilizes the resources of the central job sites, treating each one as a peer.
The increased traffic to the job sites provides new revenue opportunities for these companies.
The Real Job SearchThe Real Job Search
jobboard1.com
jobboard3.com
jobboard2.com
employer
employer
job seeker
job seeker
job seeker
internet job searching as interoperation
1 : request
2 : forward
3 : forward
2 : forward
3 : forward
4 : match