xmpp intro - the camp2012

XMPP intro & 101

TheCamp - July 2012Steffen Larsen ([email protected])

Thursday, August 2, 12

mailto:[email protected]


This PresentationI) Intro and presentation of the protocol

What is it?, Usage

Basics and architecture

Examples

II) Advanced Topics and loose talkExtensions

Plugins and components

III) XMPP 101 - Kick start and play!


About me (blah blah)

Studied Computer Science @ Copenhagen

CEO / Founder of BrainTrust ApS 2010 (Solo Ent.)

Prof. Software Developer for 15+ yearsJava (JSE / JEE), C, Erlang, Ruby, Perl.

Open Source developer.. Tigase, Strophe-plugins, XMPPConsole etc.


My XMPP Involvement

Member of XMPP Standard Foundation (XSF)

Developing XMPP since 2007 (Ms.Thesis, pidgin/VOIP)

Done XMPP design and impl. for: WAOO, Nordija (Set-top-boxes and over-the-top TV and IPTV)

Thrane&Thrane (Maritime Satellite Communication project)

Danske Spil / BetWare (Game software for Bingo etc)


XMPP Intro

What is it?

XMPP - eXtensible Message and Presence Protocol

Earlier known as Jabber - 1998

Real time messages system for communication

Routes small snipplets of XML - called stanzas


XMPP - What?Who uses it? Most of you use it every day as chat..

You? Jabber client (pidgin/adium/messenger (m$))

Google (gmail, google hangouts)

Cisco (webex)

Facebook (chat)

Even Micro$oft have XMPP gateways integrated into their messaging system


XMPP Basics

What can you do with it?

Everything that you can imagine (almost) :-)

More than just chat -- Its about presence, routing and real-time communication!

Real time web!.. its there - push it!


XMPP BasicsBasic Services and applications

Encryption / Authentication

Presence

Contact / Buddy lists

Messaging (1-1 or group chat)

Push Notifications (individual or publish/subscribe)

Service discovery and device capability advertisement

Geolocation and notifications

Voice over IP (VOIP)

Collaborative text editing etc.


XMPP applicationsSet-top box and OTT implementation done my self..

Taxi service real-time: http://cabulous.com/

Yammer.com - work in social context (ala facebook)

status.net - DISTRIBUTED twitter (micro blog) - download and federate..

JIVE sbs - big social site for business / whiteboarding

Apple mobile push

stand-alone - adium/pidgin clients etc.

Customer Service - chat and voip - call forwarding/queue

games: tic-tac toe / chess

Energy Trading software


http://cabulous.com

http://cabulous.com

XMPP BasicsWhy XMPP? just use normal HTTP (polling), or we (almost) have websockets and others..)

XMPP is an Open Standard (like HTTP) - XSF and the council does the work openly.

Proven tech. Almost 15 years of dev.

Secure - Channel encryption and strong auth.

Decentralized. no stand-alone silos (hello twitter and fb..)

Huge community

Extensibility!.. We have above 300 extensions to the XMPP core


XMPP Protocol

Core specifies (RFC 6120 / RFC 6121)

http://xmpp.org/xmpp-protocols/rfcs/

Connect with TCP or other transport protocols (HTTP/websockets, specified in extensions)

Opening a Stream tag with with streaming XML stanzas (<stanza>.. stanza xml .. </stream>)

stanzas




XMPP Architecture

XMPP can consist of many modules:Servers (many through federation)

Plugins

Components (XEP-0114)

Clients (typically many..)

Proxies

Database, LDAP, SIP and other systems (gateways)


XMPP Architecture


XMPP Addressing & Routing

Standard routing through JIDfull JID: <id>@<domain>/<resource>

bare JID: <id>@<domain> (autom. resource)

Domain routing like normal mail (smtp).


XMPP Connection Lifecycle

• Initiating TCP connection to the XMPP server

• Opening an XML <stream> for session..

• Negotiating stream features and SASL negotiation (Simple Authentiation and Security Layer)

•Auth mechanisms: PLAIN, DIGETS-MD5, SCRAM, EXTERNAL etc.

• TLS secure layer (authentication towards the server)

• The clients resource gets bound to the session

• The client sends out initial presence

• The client can now communicate (send/receive stanzas)

• The client ends session

• </stream> is send. And TCP connection is closed.

To illustrate a chat client connecting


Communication Primitives

Stanzas (xml snipplets that flow async.)

Presence

Message

IQ (Information/Query)

Simple, we only have 3 ways:


Stanzas: PresenceSets and shows the availability of the entity that is connected.

chat

away

dnd (do not disturb)

Presence for the system and the subscribers of the entity (e.g. buddylist)

Specialized Presence extension (PEP) shows what music you are playing..


Stanzas: Presence// initial simple presence<presence/>

// sending away to subscribers<presence> <show>away</show></presence>

<presence> <show>dnd</show> <status>I am working.. </status</presence>

// before logging off.. more realtime<presence type='unavailable'/>


Stanzas: Presence// direct presence

<presence from='[email protected] balcony’ to='[email protected]'/>

// Capabilities.. version, voip etc.

<presence from='[email protected]'> <c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://psi-im.org' ver='q07IKJEyjvHSyhy//CH0CxmKi8w='/> </presence>






Stanzas: MessageUsing it to communicate with other clients

TO attributefull JID: direct messaging to specific entity (e.g. device/computer).

bare JID: will route to one or more of the connected resources - depends of status, presence, message type etc.

Type attribute (chat, normal, groupchat, headline etc) - presentation and routing

Body Element - the message


Stanzas: Messagenormal message<message from='[email protected]/balcony' id='ktx72v49' to='[email protected]' type='chat' xml:lang='en'> <body>Hi romeo where art thou?</body> </message>

direct message<message from='[email protected]/balcony' id='ktx72v50' to='[email protected]/thecastle' type='chat' xml:lang='en'> <body>will you come and visit?</body> </message>


Stanzas: IQ

A more request / response model. For getting information and query the server / entities.

Request: GET / SET

Response: RESULT / ERROR

Using it for getting roster, ad-hoc command and statistics for server (if admin).. etc.


Stanzas: IQ Example 1: User requests current roster from server

UC: <iq from='[email protected]/orchard' id='hf61v3n7' type='get'> <query xmlns='jabber:iq:roster'/> </iq>

Example 2: User receives roster from server

US: <iq id='hf61v3n7' to='[email protected]/orchard' type='result'> <query xmlns='jabber:iq:roster'> <item jid='[email protected]' name='Juliet' subscription='both'> <group>Friends</group> </item> <item jid='[email protected]' name='Benvolio' subscription='to'/> <item jid='[email protected]' name='Mercutio' subscription='from'/> </query> </iq>


Now.. Lets have Coffee!

While having a break and if you want to play with XMPP later on..

Install java sdk

Install apache


II) Advanced Topics

Extensions:

Pub / Sub (XEP-0060)

BOSH (XMPP over HTTP)

External Database and LDAP

Internal routing and modifications via. plugins

Components


XMPP ExtensionsXSF defines a set of XMPP Extension Protocols (XEPs).

> 300 at the momenthttp://xmpp.org/xmpp-protocols/xmpp-extensions/.

Jingle (Gtalk), MUC (hangouts), file transfer, etc.. publish/subscribe

We will concentrate on:

publish / subscribe & BOSH (XMPP on the web)


http://xmpp.org/xmpp-protocols/xmpp-extensions/

http://xmpp.org/xmpp-protocols/xmpp-extensions/

Publish / Subscribe (XEP-0060)

Pub/Sub embbeds in IQ stanza

Put JIDs into groups and push out messages to many subscribers!

Let people subscribe/unsubscribe for data on a node

Setup roles and affiliates..



<iq type='set' from='[email protected]/blogbot' to='pubsub.shakespeare.lit' id='pub1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='princely_musings'> <item> <entry xmlns='http://www.w3.org/2005/Atom'> <title>Soliloquy</title> <summary>To be, or not to be: that is the question:Whether 'tis nobler in the mind to sufferThe slings and arrows of outrageous fortune,Or to take arms against a sea of troubles,And by opposing end them? </summary> <link rel='alternate' type='text/html' href='http://denmark.lit/2003/12/13/atom03'/> <id>tag:denmark.lit,2003:entry-32397</id> <published>2003-12-13T18:30:02Z</published> <updated>2003-12-13T18:30:02Z</updated> </entry> </item> </publish> </pubsub></iq>



<message from='pubsub.shakespeare.lit' to='[email protected]' id='foo'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='princely_musings'> <item id='ae890ac52d0df67ed7cfdf51b644e901'> [ ... ENTRY ... ] </item> </items> </event></message>

<message from='pubsub.shakespeare.lit' to='[email protected]' id='bar'> <event xmlns='http://jabber.org/protocol/pubsub#event'> <items node='princely_musings'> <item id='ae890ac52d0df67ed7cfdf51b644e901'> [ ... ENTRY ... ] </item> </items> </event></message>

Subscribers receives published data


BOSH (XEP-0124)Bidrectional streams Over Synchronous Http (BOSH)

HTTPs api are great - Polling sucks.

Long Polling. XMPP embedded into HTTP

why? simple, known tech. use compress etc.

Small problem: cross origin policy (diff. ports)

default BOSH port 5280 - HTTP 80


BOSHHTTP hold the connection until there is something

to send


BOSH - XMPP over HTTPPOST /webclient HTTP/1.1Host: httpcm.example.comAccept-Encoding: gzip, deflateContent-Type: text/xml; charset=utf-8Content-Length: 188

<body rid='1249243562' sid='SomeSID' xmlns='http://jabber.org/protocol/httpbind'>

<message to='[email protected]' xmlns='jabber:client'> <body>Good morning!</body> </message>

</body>


BOSH - XMPP over HTTP

HTTP/1.1 200 OKContent-Type: text/xml; charset=utf-8Content-Length: 64

<body xmlns='http://jabber.org/protocol/httpbind'/>

• No data - empty body tag from server• Works through header setup and timeouts


Plugins & Components


Some advanced featuresNormal client “bots” easy to do, but have limitations

To incorporate your own business logic you need some more..

To make advanced features on the server side

Server Plugins

Components

External Auth.. either plain db, ldap or others.


Server Plugins

Located on the server

Change routing strategy

Filter packages / Drop packages

Do statistics etc.

Bound to the server implementation - not portable

Can hog mem/cpu from the server. Its embedded!. (take care)


Components (XEP-0114)Components conn. to the server or vice versa (normal TCP)

Handshakes and authenticates (trusted by server)

Portable

Can be run externally with code implemented in almost every language.


Components (XEP-0114)

Acts almost like a “normal” client. e.g. JID ([email protected])...

Can act and interact on the whole domain.. not multiple

Can alter the to/from fields...

Can’t drop/filter packages for every domain.




Break!! ... Next developing XMPP 101

Lets have some hands-on experience.

Install Java (version 6 and up is fine)

Install Apache web server

Get XMPP server up and running

Play..


XMPP 101 - playtime!

Install your very own XMPP server!

Setup Apache or other httpd / proxy

Download Strophe and setup examples..

Play!


Developing XMPP

XMPP can run on every device, on mobile, on the web on

XMPP makes a bridge and makes the device present and available to communicate with other entities..

Client frameworks for almost every languageJava, Javascript, C / C++, Erlang, C#, Python, Ruby, Perl etc.

We will focus on the web (client side) and hence do javascript

Learned so far:


XMPP Kick Start!

• Choose your favorite XMPP server!

• The list is long: http://xmpp.org/xmpp-software/servers/

• Be ware of the license. Some are commercial/gpl/gpl3/apache etc.

But first! to make it work, we need a server


http://xmpp.org/xmpp-software/servers/


XMPP Servers

Administrators would look for monitoring and tools.

Extensions.. E.g. some can pub/sub, some can’t. So choose from your specific application.

Clustering or not? What is your demand? 5k users? 500k users?


My recommendations Ejabberd 2.1.11 (http://www.ejabberd.im/)

cool features dist. out-of-the-box. Thanks erlang!

high load factor

Tigase 5.1.x (linux, mac etc. *NIX.)

very flexible. Code injection on the fly. Groovy / Java. Easy to understand

Many extensions available

Good monitoring tools!Thursday, August 2, 12

http://www.ejabberd.im

http://www.ejabberd.im

Windows users?...

If you can’t run shell scripts etc.

Openfire might be easier:

http://goo.gl/9HYeb

Should be easy to setup.. I’ll help later. No worry. :-)


http://goo.gl/9HYeb

http://goo.gl/9HYeb

Tigase XMPP serverInstall Java if you dont have it!

Install tigase:

download http://goo.gl/jYNFD

run the downloaded jar file: java -jar tigase...jar

Setup easy. in step 7, choose only base, extras and derby database. in the other steps just press next!

cd {TIGASE_HOME} (where you installed it)

Start: ./scripts/tigase.sh start etc/tigase.conf

Stop: ./scripts/tigase.sh stop

add a new user to the server (use a client: PSI, Adium , Pidgin etc)


http://goo.gl/jYNFD

http://goo.gl/jYNFD

Proxy setup

because of same origin policy.. if you have a CORS enabled browser, this is not needed

http://httpd.apache.org/download.cgi

proxy_module should be enabled.

in httpd.conf (proxy port 80 to port 5280)ProxyPass /http-bind/ http://localhost:5280/http-bind/






http://localhost:5280/http-bind/

http://localhost:5280/http-bind/

XMPP libraries..

There a many! http://xmpp.org/xmpp-software/libraries/

Native or BOSH (XMPP over HTTP)

We focus on BOSH.. lets enable real-time push on the web.

Strophe.js!


http://xmpp.org/xmpp-software/libraries/


Strophe InstallDownload here: http://goo.gl/5btM2

unzip / untar the downloaded package into your Apache web folder (on mac: Library/WebServer/Documents).

You can change the folder in the httpd.conf if you want.

change in examples/echobot.js :var BOSH_SERVICE = '/http-bind/';

Ready to run!. if server is running, fire up your browser and use the user created just before when logging in.


http://goo.gl/5btM2

http://goo.gl/5btM2

Strophe

Pure javascript XMPP library (integrated with jquery)

Using BOSH as connection type

Simple manipulations with Stanzas

Build everything like DOM

Light and Event driven..

Here is my example: http://localhost/webchat


http://localhost/webchat

http://localhost/webchat

Strophe

var conn = new Strophe.connection(“localhost/http-bind”);

conn.connect(JID, password, callbackfn);

Make connection on the BOSH URL defined in apache proxy:

Connect with JID (full/bare) and make cb function:


Strophe

function onConnection () {if (status == Strophe.Status.CONNECTED) {

// do something!} else if (status .....

}

Strophe have static status to check on the connection


Strophe - Sending / Receivingconn.send(xml);....

conn.addHandler(on_message, null, “message”, “chat”);

on_message(message) { console.log(“Hi there: “ + message);

return true;}


Strophe..

1. function onMessage(msg) { 2. $(msg).find("message[type='chat'][from]:has(body)") 3. .each(function () { 4. var body = $(this).find("body:first").text(); 5. var from = $(this).attr("from"); 6. 7. log('ECHOBOT: I got a message from ' + from + ': ' + body); 8. 9. var reply = $msg({to: from, type: "chat"}) 10. .c("body") 11. .t(body); 12. connection.send(reply.tree()); 13. 14. log('ECHOBOT: I sent ' + from + ': ' + body); 15. }); 16.}


Strophe - Stanzas..

Strophe have convenience methods:

$pres(attributes..);

$msg(attributes..);

$iq(attributes..);


Strophe - Stanzas

Building a message stanza:

$msg({to: ‘[email protected]’, ‘type’ : ‘chat’}).c(‘body’).t(‘Hello zool!’);

Sending initial presence:$pres();




</stream>..the end..Contact me at

Email/XMPP: [email protected] / [email protected]

LinkedIn: http://dk.linkedin.com/in/zooldk

Illustrations by Adrian Teh (Too lazy to draw my self..)

XMPP standard foundation: http://xmpp.org

LinksServer list: http://xmpp.org/xmpp-software/servers/

Client list: http://xmpp.org/xmpp-software/clients/

Libraries: http://xmpp.org/xmpp-software/libraries/

Strophe:






http://dk.linkedin.com/in/zooldk

http://dk.linkedin.com/in/zooldk

http://xmpp.org

http://xmpp.org



http://xmpp.org/xmpp-software/clients/

http://xmpp.org/xmpp-software/clients/



xmpp intro - the camp2012

Technology