ad107 microsoft sharepoint meets ibm lotus domino

© 2011 IBM Corporation

AD107 Microsoft SharePoint meets IBM Lotus Domino: Development Deep Dive

Justin Lee | Chief Hacker | Hackerspace.sgStephan H. Wissel | LTPA | IBM Singapore

© 2011 IBM Corporation 2

Justin Lee

● Microsoft Most Valuable Professional (MVP) 2005-2011

● Software Development Consultant

● SharePoint since 2003

● Co-Founder of Hackerspace.SG

● Editor for Tech65.org

● http://justinlee.sg


Stephan H. Wissel

● Lotus Technology & Productivity Advisor

● Counsellor for personcentric development

● IBM Singapore Pte Ltd

● Blog: http://www.wissel.net/

● Twitter: notessensei

● First language: COBOL

● Linux user

● Lotus Notes since 2.1

http://www.wissel.net/


Agenda

● SharePoint overview – from a developers view

● SharePoint APIs – the moving target

● Reading and writing data from / to SharePoint

● Integrating SharePoint into XPages

● Migrating SharePoint back to Domino

● Q&A


SharePoint 2010 overview

● The business collaboration platform for the Enterprise and the Web

─ Connect and Empower People─ Cut Costs with a Unified Infrastucture─ Rapidly Respond to Business Needs



● Site─ Ribbon UI─ SharePoint Workspace─ SharePoint Mobile─ Office Client and Office Web App

Integration─ Standards Support

● Communities─ Tagging, Tag Cloud, Ratings─ Social Bookmarking─ Blogs and Wikis─ My Sites─ Activity Feeds─ Profiles and Expertise─ Organization Browser



● Content─ Enterprise Content Types─ Metadata and Navigation─ Document Sets─ Multi-stage Disposition─ Audio and Video Content Types─ Remote Blob Storage─ List Enhancements

● Search─ Social Relevance─ Phonetic Search─ Navigation─ FAST Integration─ Enhanced Pipeline



● Insights─ PerformancePoint Services─ Excel Services─ Chart Web Part─ Visio Services─ Web Analytics─ SQL Server Integration─ PowerPivot

● Composites─ Business Connectivity Services─ InfoPath Form Services─ External Lists─ Workflow─ SharePoint Designer─ Visual Studio─ API Enhancements─ REST/ATOM/RSS


SharePoint Application Landscape

● SharePoint Core─ .NET + ASP.NET

● Business Collaboration Applications

● Single WebPart Solutions

● Customized SharePoint Server 2010 Workloads

● Portal for LOB Application Data


SharePoint Overview - Developers

● Developer Productivity─ Rapidly develop solutions with Visual Studio 2010─ Application Life-Cycle Management & Team Development ─ Solution quality improved with debugging tools

● Deployment Platform─ Rich solution UI with Silverlight, XSLT View, Clean HTML ─ Data Foundation – Relationships, Lookup─ Better programmability with LINQ, Client API

● Solution Hosting─ Standardized WSP Package Deployment─ SharePoint Online shared solution hosting─ Install on Vista SP1 or Windows 7 for Developers only


Comparing Notes & SharePoint


The core of SharePoint development: Lists

● Collaborative Data Store based on Lists─ Relationships with Cascade or Block─ Transacted Cascading Deletion─ Restriction of Deletion

● Validation with Excel Like Formula─ Formula-based Validation─ Ensure Uniqueness

● Lookup to Multiple Columns

● List Index Auto-Creation

● Scalability and Performance

● List Query Throttling

● Self Service List Creation

● Out of the Box List User Interface

● Built in Authorization Forms


XSLT Views for Lists

● Replaces CAML for Views─ CAML Still Used for Queries

● Better Performance

● Easier edit in SharePoint Designer─ Ribbon UX: sort/filter/group, conditional formatting,…

● Simpler Coding Experience─ Standard language, shared default XSLT

● XSLT-based views are now default─ SharePoint Lists─ External Lists


Examples of Lists

● Document Libraries storing Word Documents

● Page Libraries storing Pages in a Publishing Portal

● Tasks Libraries storing Tasks in a Workspace

● Custom lists created by users


Agenda

● SharePoint overview – from a developers view

● SharePoint APIs – the moving target

● Reading and writing data from / to SharePoint

● Integrating SharePoint into Xpages

● Migrating SharePoint back to Domino

● Q&A


SharePoint API

● 2003─ Web services WS-*

● 2007─ Web services WS-*

● 2010─ Web services WS-*─ REST + ATOM → oData


WebServices-*

Name URL DescriptionAdministrative methods such as creating and deleting sites

[Alerts] Methods for working with alerts[Copy]

Methods for retrieving schemas and data[DWS] Methods for working with Document Workspaces[Forms] Methods for working with user interface forms[Imaging] Methods for working with picture libraries[Lists] Methods for working with lists[Meetings] Methods for working with Meeting Workspaces[Permissions]

[Sites] Contains a single method to retrieve site templatesMethods for working with users and groups

[Versions] Methods for working with file versions[Views] Methods for working with views of lists

Methods for working with Web Parts[Webs]

[Admin] _vti_adm/Admin.asmx_vti_bin/Alerts.asmx_vti_bin/Copy.asmx Methods for copying to and from SharePoint sites

[DspSts] _vti_bin/DspSts.asmx_vti_bin/DWS.asmx_vti_bin/Forms.asmx_vti_bin/Imaging.asmx_vti_bin/Lists.asmx_vti_bin/Meetings.asmx_vti_bin/Permissions.asmx Methods for working with SharePoint Services security

[SiteData] _vti_bin/SiteData.asmx Methods used by Windows SharePoint Portal Server_vti_bin/Sites.asmx

[UserGroup] _vti_bin/UserGroup.asmx_vti_bin/versions.asmx_vti_bin/Views.asmx

[WebPartPages] _vti_bin/WebPartPages.asmx_vti_bin/Webs.asmx Methods for working with sites and subsites

*

* That much is available (for now in SP2010) as oData)


oData – not “oh some Data”

● Open Standard─ SAP─ Websphere─ http://www.odata.org

● Powered by ADO.NET Data Services "Astoria"─ REST Protocols: Atom, AtomPub, and RSS

● Syntax:─ /_vti_bin/ListData.svc/{Entity}[({identifier})]/[{Property}]

● Example to get budget hours for Project $4:─ /_vti_bin/ListData.svc/Projects(4)/BudgetHours

● Example to get Projects for Clients in Chicago:─ /_vti_bin/ListData.svc//Projects?$filter=Client/City eq 'Chicago'

● Example to get a Project and its related Client:─ /_vti_bin/ListData.svc/Projects?$expand=Client─ Shows me parent that is associated with this XML


oData in SharePoint

● QueryString parameters for REST─ $filter─ $expand─ $orderby─ $skip─ $top─ $metadata (will bring back all the XML metadata about the object. Think of it like

WSDL for your REST call)─ You can stack these parameters


Agenda

● Sharepoint overview – from a developers view

● Sharepoint APIs – the moving target

● Reading and writing data from / to Sharepoint

● Integrating Sharepoint into Xpages

● Migrating Sharepoint back to Domino

● Q&A


Reading and writing data

● 2003: Pick a tool for web services

● 2007: Pick a tool for web services (I use oXygen XML)

● 2010: Curl is just fine

Download CURL here: http://curl.haxx.se/download.html

Basic syntax (for our purpose):

curl -ntlm -netrc http://sharepoint/_vti_bin/[moreurl]

.netrc (mode 600)machine sharepoint login user password pwd

http://curl.haxx.se/download.html

http://sharepoint/_vti_bin/%5Bmoreurl

http://sharepoint/_vti_bin/%5Bmoreurl


Reading and writing data

● Reading a subset of data:─ build a view─ use a CAML query

● CAML─ CAML Viewer: http://spcamlviewer.codeplex.com/─ CAML Builder: http://www.u2u.net/res/Tools/CamlQueryBuilder.aspx

● Paging data─ <rs:data ItemCount="25" ListItemCollectionPositionNext="Paged=TRUE&p_ID=25">─ <rs:data ItemCount="25" ListItemCollectionPositionNext="Paged=TRUE&p_ID=50">─ ListItemCollectionPositionNext missing if no more records!

─

http://spcamlviewer.codeplex.com/

http://www.u2u.net/res/Tools/CamlQueryBuilder.aspx


Web Services*

* it always starts with WSDL


SharePoint WSDL

● http://sharepoint/_vti_bin/Lists.asmx?WSDL

● 3 of 10 name spaces propriety (actually quite open)

● Potential duplication of wdsl data types (name space s / s1)?


SharePoint WSDL

● 2 Ports: ListsSoap, ListsSoap12

● > 1700 lines of wsdl

● 32 operations

… more later


Reading: WebServices

● http://sharepoint/_vti_bin/Lists.asmx

● GetListCollection ()─ Returns a collection of all the Lists─ http://msdn.microsoft.com/en-us/library/lists.lists.getlistcollection(v=office.12).aspx

● GetList (string listName)─ Returns the List fields schema─ http://msdn.microsoft.com/en-us/library/lists.lists.getlist(v=office.12).aspx

● GetListItems (string listName, string viewName, XmlNode query, XmlNode viewFields, string rowLimit, XmlNode queryOptions, string webID)

─ Returns items in the list─ http://msdn.microsoft.com/en-us/library/lists.lists.getlistitems(v=office.12).aspx


Updating: WebServices

● http://sharepoint/_vti_bin/Lists.asmx

● UpdateListItems (string listName, XmlNode updates)─ For items successfully updated, a row fragment is returned with the updated row

values.─ http://msdn.microsoft.com/en-us/library/lists.lists.updatelistitems(v=office.12).aspx


Web Service caveats

● Requires NTLM authentication

● More than one Service end point is needed─ List: get the available lists─ View: get the views for a list─ List: Get data for List/View combination─ Data: get all data for a List

● Different terminology─ Sharepoint List = Notes Document Collection─ Sharepoint View = Notes View Search Result


REST / ATOM / oDATA*

* URLs are just fine


Reading data: GET

curl --ntlm --netrc http://sharepoint/_vti_bin/ListData.svc/Calendar

Watch this


Reading data: GET (2)


Watch this


Reading data: GET (3)


Soap data type duplication! (modeled after oData.org)


Creating data: POST (send JSON, get oDATA)

curl –ntlm --netrc

-d "{ Title: 'January Team Meeting', StartTime: '2011-01-15T17:00:00', EndTime: '2011-01-15T18:00:00' }"

-H content-type:application/json

-X POST http://sharepoint/_vti_bin/ListData.svc/Calendar


Creating data: POST


-d "{ Title: 'January Team Meeting', StartTime: '2011-01-15T17:00:00', EndTime: '2011-01-15T18:00:00' }"


-X POST http://sharepoint/_vti_bin/ListData.svc/Calendar


Modifying data: PUT the whole new version


-d "{ Title: 'January 2011 Team Meeting', StartTime: '2011-01-15T17:00:00', EndTime: '2011-01-15T18:00:00'}"


-H If-Match:*

-X PUT "http://sharepoint/_vti_bin/ListData.svc/Calendar(2)"

PUT recreates the entry.

If-Match Header is for Concurrency Management


Modifying data: writing only changes


-d "{ Title: 'January 2011 Team Meeting'}"


-H If-Match:*

-X MERGE "http://sharepoint/_vti_bin/ListData.svc/Calendar(2)"

MERGE modifies just that entry.

If-Match Header is for Concurrency Management.


Deleting data: URL as key

curl --ntlm --netrc

-X DELETE "http://sharepoint/_vti_bin/ListData.svc/Calendar(2)"

If the operation executed successfully servers should return 200 (OK) with no response body


Agenda




● Integrating Sharepoint into XPages


● Q&A


Integrating SharePoint into XPages

● 2003, 2007: Web services only

● 2010: oData

● Design as Data Source (see also SHOW107)

● Tools:─ Apache Axis, Apache CXF (Web service)─ LotusScript agents? (for occasional queries)─ Restlets (oData) http://www.restlet.org/─ oData4J (oData) http://code.google.org/p/odata4j

(Yes we know – it is almost all Java stuff)

─ There is some JavaScript: http://www.odata.org/Developers/Articles#Javascript─ A Dojo data store is in the works

http://www.restlet.org/

http://code.google.org/p/odata4j

http://www.odata.org/Developers/Articles#Javascript


Managed Bean as Data Source

● calls:getDesertName()setDesertName(new)


Demo


Agenda






● Q&A


Migrating back to Domino

● Consider the target─ Domino─ Quickr─ Connections

● Read the lists

● Read the list structure

● Generate the XPages, forms and documents

● Move the data

● Nope – no business logic


Migrating back to Domino

● Caveats:─ Domino web service client doesn't support NTLM

use Apache Axis instead─ Most likely you will migrate SP2003 / 2007, so oData is out─ Prepare Sharepoint Views to accelerate migration

● Considerations:─ Use the OneUI─ Coexistence phase: surface Data in Xpages─ Command line tooling for migration allows more threads / memory─ Export can be performed many times─ “Hot” export could write back “Migration DateTime” into individual data item as well

as new Notes/Domino URL─ List can be combined since there are less structure and space limitations


Read the lists

● http://sharepoint/_vti_bin/Lists.asmx<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://schemas.microsoft.com/sharepoint/soap/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <ns0:GetListCollection/> </SOAP-ENV:Body></SOAP-ENV:Envelope>

● Results:─ 4 payload fields─ 54 internal fields─ > 1200 LOC

http://sharepoint/_vti_bin/Lists.asmx

http://schemas.xmlsoap.org/soap/envelope/


http://schemas.microsoft.com/sharepoint/soap/



Read the lists

● http://sharepoint/_vti_bin/Lists.asmx<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://schemas.microsoft.com/sharepoint/soap/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <ns0:GetListCollection/> </SOAP-ENV:Body></SOAP-ENV:Envelope>

http://sharepoint/_vti_bin/Lists.asmx






Read the list structure

● <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://schemas.microsoft.com/sharepoint/soap/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <ns0:GetList> <ns0:listName>{28B5B773-9FE2-49F8-B71C-65E12F35A04E} </ns0:listName> </ns0:GetList> </SOAP-ENV:Body></SOAP-ENV:Envelope>






Read the list structure

Clearly a case for XSLT(wait for the Demo)

Caveat: List structure != returned data!!!!


Read the views

● <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://schemas.microsoft.com/sharepoint/soap/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <ns0:GetViewCollection> <ns0:listName>{299EC8EC-9CE5-4225-838C-7066A4541B43}</ns0:listName> </ns0:GetViewCollection> </SOAP-ENV:Body></SOAP-ENV:Envelope>






View Results


Generate Design Elements

● DEMO


Move the data

● <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://schemas.microsoft.com/sharepoint/soap/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <ns0:GetListItems> <ns0:listName>{28B5B773-9FE2-49F8-B71C-65E12F35A04E}</ns0:listName> <ns0:viewName>{C98EEC9D-D615-4E82-970E-B509169035AA}</ns0:viewName> <ns0:query/> <ns0:viewFields/> <ns0:rowLimit>42</ns0:rowLimit> <ns0:queryOptions/> </ns0:GetListItems> </SOAP-ENV:Body></SOAP-ENV:Envelope>






Demo


Agenda






● Q&A


Q & A


Legal Disclaimer© IBM Corporation 2011. All Rights Reserved.

The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.

All references to refer to a fictitious company and are used for illustration purposes only.