worldos corp

WorldOS Corp.WorldOS Corp.

NYJavaSig Talk 1/30/2001

IntroductionIntroduction

Me– Lucas Gonze– Web/eCommerce developer– Open source culture– Network emphasis


WorldOS– One year old– Topology centric– Java– About the software– About the business


Web era led to Java and Windows standoff

In general Internet era, what next?– .Net & C#?– Java on the client?– Client->Server or Server->Client?

• Whither Perl, Python and VB?

OverviewOverview

This talk will– Introduce peer to peer ideas– Evaluate Java as a tool for peer to peer.– Show common approaches– Show infrastructure you can use

Focus on practical knowledge over theory.

Basics of Peer to PeerBasics of Peer to Peer

Working definition of P2P: Projects that followed directly on Napster.

P2P networks often allow computers to act as both client and server.

Every computer connected to the network will probably provide resources to the system as a whole. That is not as true for a web browser.

Basics of Peer to PeerBasics of Peer to Peer

You only own your side of the connection. The other side is a black box.– It is not safe to assume that you wrote the software on the

other side. You have no way of knowing what is on the other end of a connection.

– This precludes approaches like storing unsigned state in outgoing messages, and assuming that it is canonical when it comes back.

– Use nodewise encapsulation to leverage this heuristic.

Connections between nodes are interfaces, not buses. Design for rough grained transactions.

Client/Server Job SearchClient/Server Job Search

jobboard1.com

jobboard3.com

jobboard2.com

employer

employer

job seeker

job seeker

job seeker

Interconnections must be mediated

P2P Job SearchP2P Job Search

jobboard1.com

jobboard3.com

jobboard2.com

employer

employer

job seeker

job seeker

job seeker

1 : request

2 : forward

3 : forward

2 : forward

3 : forward

4 : match

Interconnections may be unmediated

Families of P2P applicationsFamilies of P2P applications

Put “dark (but normal) matter” to work– CPU

• Seti@home, PopularPower

– Files• Napster, Gnutella

– Bandwidth• Swarmcast

Families of P2P applicationsFamilies of P2P applications

Use the unique properties of decentralized computer networks– Privacy

• Publius

• Freenet

– Complex systems • MojoNation

• Random number generator

• Economic modeling

• Cellular automata

An Example: SwarmcastAn Example: Swarmcast

Parallel streams via rebroadcast Immunizes against /. effect. As a piece of

content becomes more popular, downloads become faster to the consumer, and less costly to the provider.

Flood strategy. You can’t be inefficient in using stuff that would have been thrown out anyway.

HeterogeneousnessHeterogeneousness

P2P puts network effects to workPutting network effects to work means

large networksIn a very large network there is no boss

to enforce standardsTo thrive without standards:

– Use LCD approach– Be adaptable

HeterogeneousnessHeterogeneousness

Don’t try to force it – the network is bigger than you.

Jujitsu is the only chance you have.Think of the Fed manipulating the US

economy via the prime rate, rather than by wage and price controls.

Heterogeneousness & JavaHeterogeneousness & Java

Evaluating the suitability of Java to P2P is a question of how well Java helps you to exploit network effects.

A Java program is often poorly adapted to local conditions like “users are not computer experts”. A native program can be optimized for a dominant platform. This allows it to

– target and take advantage of local conditions (e.g. with a newbie oriented GUI), and

– leverage the ensuing popularity to drive native implementations on other platforms.

– Such a program thrives by avoiding heterogenous situations. Not a bad strategy!

So how does Java stack up in taking advantage of network effects? That is, if you are are trying to use jujitsu to get network effects on your side, how does Java measure up the problem?

Heterogeneousness & JavaHeterogeneousness & Java

Strategies: Avoid heterogeneousness

– Example: Microsoft applications software works extremely well, but only in Windows

Stick to least common denominators– Example: Email works everywhere, for everyone, but only if

we all stick to the standards at the time email became popular - ASCII and SMTP.

Be as adaptable as possible– Example: A P2P project with an IE plugin that allowed

searching Gnutellanet, Napster and Google at the same time.

Java’s ability to exploit LCDsJava’s ability to exploit LCDs

HTTP– Pro: Jakarta-Tomcat works great– Con: Tomcat requires Java 2 or better

ASCII– Pro: Java’s support for ASCII is second to none– Con: …

Java is itself designed to be an LCD– Pro: JVMs < Java 2 are common on user machines– Con: JVMs >= Java 2 are rare on user machines

Windows– Pro: popularity of Mac and Unix.– Con: ubiquitousness of Windows.

Java’s ability to exploit LCDsJava’s ability to exploit LCDs

Conclusion: If Java 2 isn’t needed Or if your users will download the JRE(*) Or if your app needs to support Mac and Unix

usersThen Java is a good bet for exploiting LCDs.

*broadband makes this reasonably likely for high end users; dialup makes this unlikely for consumer users.

Java’s ability to be adaptableJava’s ability to be adaptable

Multiprotocol– Pro: Java’s roots among network oriented programmers (Unix people)

make it the killer environment for TCP development.• There exist excellent implementations of

– SMTP (via Javamail)– NNTP– HTTP– Gnutella– CORBA– RMI– SOAP/HTTP

– Con: No support for raw IP. • No implementation of “Ping”!

Why does multiprotocol matter? Network effects of joining multiple subnets.


Hardware agnostic – handles multiple binary formats gracefully– Pro: Network-order byte stream everywhere

– Con: C has better tools for binary operations.

– Con: RMI is Java-to-Java only. WORA

– Pro: not bad, all things considering. No serious competition in this area, because no other interpreted languages have the same range of support.

– Con: not nearly good enough.


Flexible trust models– Pro: Huge range of encryption, AI and other modern

tools.– Pro: Fine grained sandboxing policies.– Pro: Runtime loading and evaluation allow great

flexibility.– Pro: Java’s user base in the complex systems academic

communities– Con: CPU intensive work plays to Java’s weakness

(performance).– Con: MojoNation is the most mature toolkit, and that is

in Python. WorldOS is in Java, but is far from mature.


XML – Pro: Allows for decentralized extension of message formats.

– Pro: Is unequalled cross-platform glue.

– Con: Has no efficient binary equivalent for high-volume messaging (yet).

Java XML support

– Pro: Industry standard XML tools are Java – Xerces, Xalan, JDOM.

– Con: Java 2 required for Xerces, Xalan, JDOM.

– Con: Parsers are slow.


Conclusion:

If high adaptability is needed, Java is a good tool.

Real World ComparisonsReal World Comparisons

Gnutella– not Java: Bearshare

• Footprint 1.09 MB• Appearance pretty great

• Usability excellent throughout– (but partially due to much effort)

– Java: LimeWire• Footprint 6.39 MB

• Appearance somewhat ugly

• Usability relatively bad – (but not due to not trying)

– Does not match other native software

• e.g. the file open and save dialogs are not the standard

• E.g. you can’t sort a file list by clicking on sub-heads

– Buggy

• E.g. the “details” option in the file browse dialog doesn’t work

Real World ComparisonsReal World ComparisonsGnutella/Windows: BearShare

Real World ComparisonsReal World ComparisonsGnutella/Java: LimeWire

Real World ComparisonsReal World Comparisons

Unique features of Limewire– Community -- Enter an open community of similar users, as

defined by interests and computing power / bandwidth, to optimize search performance

– Anti-Freeloader Features - Choose to protect and enhance the gnutella network by "preferring" nodes who share files and information.

– Browse Host -- See all files a given host has – Library Function -- Launch and manage downloaded and

shared files – Relevancy Scoring -- See which search results are most

relevant to the search terms

ImplementationImplementation

Q: How do you get started?A: Look for precedents.

Implementation: Client+ServerImplementation: Client+Server

Simplest approach to converting a web app to a P2P app is to run the web server on localhost.– Minimal modifications to the code, if

you’re lucky.– Need a database, tag replacer, perl etc.

on client side.– Immature technology.

Implementation: Implementation: Desktop Web ServerDesktop Web Server A web server running on the client Webby UI

– Either • SOAP calls between native GUI and your web server.

– Interface can be written in VB

– Magi uses this approach

– OR– HTTP calls between browser UI and your web server

– With reasonable latency web UIs are much more usable.

Implementation: Implementation: Desktop Web ServerDesktop Web ServerDesktop web servers

– Pro:• HTTP and HTML are mature designs.• Can use web skills and tools.• With home and UI uncoupled, you have

remote access to the app.• Lower fixed costs to pay for centralized

servers.

Implementation: Implementation: Desktop Web ServerDesktop Web Server Desktop web servers

– Con:• You have to make a web server look and feel like a GUI

application.• Immature technology – you have to roll your own.• Security – how to avoid opening up the user’s machine to

crackers?• You still have to put up with web problems like browser

incompatibility and statelessness.• OS may not be well adapted

– Scheduling problems– DNS lookup of localhost when offline

• If everyone uses this approach then server sockets will be crowded.

Implementation: Implementation: Desktop Web ServerDesktop Web Server

– Java web servers:• Sun Brazil

– small and adaptable

• Apache Jakarta-Tomcat– Reliable, large, Java 2

• Apache Jserve– Reliable, large, Java 1

• Roll your own– Best fit, time consuming, you know the code

Implementation: Implementation: PublishingPublishing Like Freenet, MojoNation, FreeHaven Allows for remote disk storage. Splits data up across a number of different

computers. Heavy encryption and security demands. Use Freenet for a Java base See Uprizer for a commercial

implementation and professional services.

Implementation: Implementation: Data SynchronizationData SynchronizationAllows for synchronizing data across

multiple devices.For example, sharing your schedule

across your laptop, PDA, PC and cell phone.

To avoid reinventing the wheel WRT version control, use CVS or WebDAV

Use Magi for an Java base

Implementation: Implementation: InfrastructureInfrastructureOpen source Java projects that you

can work from:Publishing Freenet

Bandwidth Sharing OpenCola->Swarmcast

Decentralized FileSharing Furi

Hub and Spoke File Sharing Jnap (http://www.perham.net/mike/jnap/)

Desktop Web Server Tomcat, Jserve, Brazil

Security WorldOS

Adaptability Brazil, WorldOS

Data Synchronization Magi

SummarySummary

Use – Nodewise encapsulation– Rough grained transactions– Either LCDs or Adaptability– Java 2 for broadband users, Java < 2 if

your users are dialup.Take advantage of network effectsUse jujitsu

WorldOS Application EngineWorldOS Application Engine

The barebones skeleton of a peer node. A data interchange Plugin points for

– Protocol handlers

– “Peer CGI” (P-CGI) scripts

System ArchitectureSystem Architecture

Protocol MorphingProtocol Morphing

A protocol agnostic stance for transport and message protocols. This allows developers to use any collection of protocol that best fits their needs, enabling Goa users access to take advantage of resources on other networks.

Hence, a Goa user searching for a file could simultaneously query Napster, MojoNation, and Lightshare via a single user interface.

Protocol MorphingProtocol Morphing

Protocol morphing leverages the externalities of multiple networks, creating a ‘supernetwork’ for Goa users.

P2P applications

not to scale

MojoNation

Lightshare

NapsterWorldOS Goa

P-CGI Application InterfaceP-CGI Application Interface

Traditional Client/ServerTraditional Client/Server

Information and processing are distributed on the Internet, but much of the control [applications] is still greatly centralized.

internet

client

client

client

server

client/server network

P2P Peer NetworkP2P Peer Network

Peer technology blurs the distinguishing line between clients and servers, creating a network of peers, such that each node can behave as both client and server.

internet

peer

peer networkpeer

peer

peer

peer

Peer to Peer (P2P)Peer to Peer (P2P)

P2P networks allow computers to act as both client and server.

Each computer connected to the network provides resources to the system as a whole.

P2P is particularly useful for scalability and redundancy. Use of P2P to reduce single points of failure and controller bottlenecks is likely to be effective.

Data centers and Akamai-type mirrors are natural candidates for P2P.

The Status Quo Job SearchThe Status Quo Job Search

Client-Server Technology

A job seeker has to log onto multiple job sites, post resumes and search for jobs.

An employer must go through the same steps, seeking out relevant resumes and posting jobs.

The Status Quo Job SearchThe Status Quo Job Search

jobboard1.com

jobboard3.com

jobboard2.com

employer

employer

job seeker

job seeker

job seeker

internet job searching in the status quo

The Ideal Job SearchThe Ideal Job Search

The Ideal – Peer-Peer TechnologyA single request can be sent from a peer to its neighbors, being forwarded [via viral distribution] along to others. Hence, the success rate for a prospective employer or employee has a very high probability of success with minimal effort.

The Ideal Job SearchThe Ideal Job Search

employer

employer

job seeker

job seeker

job seeker

internet job searching in a peer network

job seeker

The Real Job SearchThe Real Job Search

Leveraging The Strengths Of Each System

This network is interoperable with currently installed systems and utilizes the resources of the central job sites, treating each one as a peer.

The increased traffic to the job sites provides new revenue opportunities for these companies.

The Real Job SearchThe Real Job Search

jobboard1.com

jobboard3.com

jobboard2.com

employer

employer

job seeker

job seeker

job seeker

internet job searching as interoperation

1 : request

2 : forward

3 : forward

2 : forward

3 : forward

4 : match

worldos corp

Documents