blip basic lightweight information protocol philosophy, design decisions, and experimental results...
TRANSCRIPT
BLIPBLIP Basic Lightweight Information ProtocolBasic Lightweight Information Protocol
Philosophy, Design Decisions, Philosophy, Design Decisions, and Experimental Resultsand Experimental Results
Matt Jensen
blip.org
www.blip.org mailing list: [email protected]
FOR MORE INFO...
BLIP DescriptionBLIP Description
A federated, server-based protocol for near real-time, robust, scalable, subject-based publish-and-subscribe messaging. Useful for user-oriented or system-oriented communication.
Project GoalsProject Goals
Open standard for P&S messagingPrototyping and experimentingAggressive timing goals
Project GoalsProject Goals
Open standard for P&S messaging– Platform and Language Neutral
• run anywhere• handhelds to enterprise server farms
– Simple, text-based wire protocol– Leverage existing standards– Robust, can be extended
Project GoalsProject Goals
Prototyping and experimenting– Provide freeware tools– Provide source code– Let a thousand flowers bloom
Project GoalsProject Goals
Aggressive timing goals– Early tools available now– Finalize first-phase protocol in Aug.– End user tools by September– Fold lessons learned into next standard
OriginsOrigins
Limitations of “push”
OriginsOrigins Limitations of “push”
– No open standard; big buy-in, ads– Little control over info sent– Little control over notification techniques– Polling is slow, congests corporate network– Synchronous - miss messages when offline– Limited user-created topics– Limited data formats– Limited security, reliability
OriginsOrigins
Requirements for a Good System
OriginsOrigins
Requirements for a Good System– Open system– Notification is the essence– User has fine-grained control over what they receive, when, and how.– Publishers can send any MIME type data.– Persistent messages needed (offline, etc.)– User accounts needed
• Publishers want to control access• Subscribers need to keep track of what they’ve seen
OriginsOrigins
Formed blip.org – Non-profit– Encourages freeware– New web site, mailing list
OriginsOrigins Moving beyond Push
– General Notification• Buddy lists/collaboration tools• Printers• System monitoring, security
– Reliability• Once, only once, in-order, guaranteed delivery• Support transactions• You’re already 90% there• Opens new opportunities
BLIP - Possible ApplicationsBLIP - Possible Applications
Apps for People– News– Buddy lists– Monitoring (email, web pages)– Security (where’s my kid, my car?)– Persistent chat– Auctions– Stock trading (personal program trading)
BLIP - Possible ApplicationsBLIP - Possible Applications
Apps for Organizations Intranet/Extranet news delivery Custom business events Track inventory levels Workflow apps (e.g., DAV) Transactions
Process orders, coordinate databases, financial transactions
Distributed/parallel processing
ReliabilityReliability
Publish & Subscribe
+ Reliable Message Queueing
+ Open Internet Protocol
= BLIP
Reliability - Message QueueingReliability - Message Queueing
– Uses• Tie systems together
– distributed processing– financial transactions– only send news if you can send bill, too
• Personal Assurance– personal apps can be mission-critical
Reliability - Message QueueingReliability - Message Queueing– Uses
• Tie systems together– distributed processing– financial transactions– only send news if you can send bill, too
• Personal Assurance– personal apps can be mission-critical
– Implementation Costs• Add two-phase commit• Optional
– All servers must support it– A topic can turn it off
Reliability - Existing ToolsReliability - Existing Tools
– Publish & Subscribe• Proprietary tools• Many new Java companies• No interoperability or openness
– Message Queueing• Very proprietary (MQSeries, MSMQ, etc.)• Slow efforts at cooperation (BMQ)
Reliability - JMSReliability - JMS
– Java Messaging Service (JMS)• Provides event services• Limitations• Sun-directed; not open ?• Java only
JMS JMS
Java Client Java Server
BLIP BLIP
JMS
Reliability - JMSReliability - JMS
– Java Messaging Service (JMS)• Provides event services
• Limitations
• Sun controlled - not open
• Java only
• JMS on top of BLIP/other standard?
JMS is an API for Java, which could use BLIP/other
as underlying protocol.
JMS
Java Client Java Server
BLIP BLIP
Other API Other API
Java Client Java Server
Reliability - JMSReliability - JMS
– Java Messaging Service (JMS)• Provides event services
• Limitations
• Sun controlled - not open
• Java only
• JMS on top of BLIP/other standard?
BLIP BLIP
Other API Other API
Java Client Java Server
public interface SimpleBlipAwareness { public void fireBlipMessageReceived(
int aTopicID, int aMessageID, String headline, String completeMessage);
public void fireBlipError(String e); public void fireBlipConnectionStateChange(
int newState, int oldState);}
Reliability - DatabasesReliability - Databases
Very useful for persistent messages– Offloads queries from event server– Leverage existing tools
• SQL for queries• Proven reliability -- transactions, recoverability• High performance (depending on domain)• Integration w/ other systems
– Middleware community moving this way
DB
Reliability - DatabasesReliability - Databases
Not required everywhere– Perceived as slower– Power not always needed
DBServer
Custom Filing System
DBTopicMgrDBTopicMgr
SimpleTopicMgrSimpleTopicMgr
Topics
Reliability & PeopleReliability & People
People want notification systems that are reliable.
“Almost everything can be mission critical to someone.”
BLIP Software - CurrentBLIP Software - Current
Java server Java client classes Java applets Win95 native client Perl send/receive scripts Simple client OCX VB demos
BLIP Software - UpcomingBLIP Software - Upcoming
Palm OS client WinCE client Native Linux server Native NT server More applets Bridges to MSMQ, MQSeries ? Possible JMS wrapper ?
The BLIP ProtocolThe BLIP Protocol
The BLIP ProtocolThe BLIP Protocol
What BLIP doesn’t do
– Presentation• topic or client specific
– ACLs• looking at other standards
– Topic discovery (?)• ACAP/LDAP
– Advanced security• Open hook to other systems
The BLIP Protocol - The BLIP Protocol - SubscriptionsSubscriptions URL-based ACLs are out of band Different cases:
– messages from now on– messages from #1 on– start at 10 messages ago
Default policies– e.g. - buddy lists
The BLIP Protocol - MessagesThe BLIP Protocol - Messages
SMTP-style headers
Any MIME type data– text– GIF– HTML– XML– etc. ...
The BLIP Protocol - Filters?The BLIP Protocol - Filters?
Server-side rules– XML, but how to script?
The BLIP Protocol - Filters?The BLIP Protocol - Filters?
Server-side rules– XML, but how to script?
Currently, repackage data as new topic– like “Event Refinement” in Keryx ?
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– Improves scaling– Offers levels of service
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– ALIVE
ClientServer
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– ALIVE
ClientServer
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– ALIVE
ClientServer
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– ALIVE
ClientServer
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– AWARE
ClientServer
tag
IP
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– ALIVE
• Fast - maintains connection• Expensive - maintains connection
– AWARE• Slower - create TCP connection• Scalable - only use needed connections
The BLIP ProtocolThe BLIP Protocol
Multiple access modes– ALIVE
• Fast - maintains connection• Expensive - maintains connection
– AWARE• Slower - create TCP connection• Scalable - only use needed connections
– Other modes?• Multicast -- Reliable Multicast very delicate
– RFC 2357
The BLIP ProtocolThe BLIP Protocol
Modes allow flexible scaling– High-priority users get ALIVE mode
• ISPs, paying subscribers
– Low-priority users get AWARE mode
Server
ALIVE
AWARE
The BLIP ProtocolThe BLIP Protocol
Another example
Server “Real” Clients
200 ALIVE clients per level3 levels...= 8 million ALIVE clientswith 5-10 second delay(?)
can it be coordinated?
Proxies/Relayers
The BLIP Protocol - Client StatesThe BLIP Protocol - Client States
Unauthorized Ready
Receiving
Sending
Login
The BLIP Protocol - Client StatesThe BLIP Protocol - Client States
Unauthorized Ready
ReceivingSending
C: Acknowledge
S:
Tra
ns
ac
tio
nC: Acknowledge
S:
Tra
ns
ac
tio
n
C: Acknowledge
S:
Tra
ns
ac
tio
n
S: UPDATE TXN=1S: START ITEM #322 61S: Headline: Inflation is upS: Posted 05 Jul 1998 12:49:55 GMTS: Content-Length: 145S:S: The Federal Reserve. . .S: END ITEM (additional items)S: END UPDATES:C: !A002 ACK 1C:S: !A002 200 - OK.S:
The BLIP Protocol - A SessionThe BLIP Protocol - A SessionS: BLIP 0.46 MODES=ALIVES:C: LOGIN joeuser mypassword mode=ALIVEC:S: 200 - OK. Logged in.S:C: !A001 UPDATES ONC:S: !A001 200 - OK.S:
either immediately, or after some time…
The BLIP Protocol - VariationsThe BLIP Protocol - Variations
S: BLIP 0.46 MODES=ALIVE
S: BLIP 0.46 MODES=ALIVE,AWARE
C: LOGIN joeuser mypassword mode=ALIVE
C: LOGIN joeuser mypassword mode=ALIVE,AWARE
C: LOGIN joeuser mypassword mode=ALIVEC:
C: LOGIN joeuser auth=XYZ mode=ALIVEC: Key=de5te4545665r65e4ddhkjdkasiudy753sdC:
Modes - Server variations
Modes - Client variations
Security - Client variations
Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point
BLIP is server-based– Not point-to-point– Could be used w/ p-to-p tools
Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point
With few clients, p-to-p is nice– reduces server load– can reduce latency– could enhance privacy
With many clients, client gets burdened– Needs sophisticated programming– Especially for real-time collaboration
Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point
Example - Spot Demo
Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point
Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point
Old user sees this... New user sees this...
Clients
Del
ay (
ms)
0
50
100
150
200
250
300
350
400
450
500
2 10 20 30 40
Point-to-Point
Client/Server
Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point
IssuesIssues
Roaming users and AWARE mode– multiple ‘active’ client devices per
person?• “Location Server” from SIP?
(Session Initiation Protocol)
IssuesIssues
Do Subscribers get messages sent before their subscription started?
– “Quenching”– Multicast
– Flexibility
IssuesIssues
Dynamic filter control– How to fine-tune what you receive, on
the fly, without disturbing subscriptions?
C: !A0002 UPDATES ONC: - /nasdaq/quotes/*C: + /nasdaq/quotes/msftC: