using service-oriented methodologies to create sharepoint applications by eugene rosenfeld -...

www.BlackBladeInc.com | [email protected] | +1-703-260-1111

Using SOA to Create Using SOA to Create SharePoint ApplicationsSharePoint ApplicationsThis presentation focuses on applying the methodologies and architectural concepts of Services Oriented Architecture (SOA) to the task of creating complex SharePoint applications. We will see how a SOA approach can help us achieve a stable and scalable application or product, decrease support and maintenance costs by decreasing the application's complexity, and reduce the effort required to upgrade the application to the next version of SharePoint.

Eugene RosenfeldCTO, Black Blade Associates, Inc.


About Eugene Rosenfeld CTO, Black Blade Associates Two-time SharePoint MVP

www.BlackBladeInc.com twitter.com/erosen03

ThingsThatShouldbeEasy.blogspot.com [email protected]


About Black BladeSpecialize in SharePoint document and records management, information architecture, distributed systems, and systems integration

www.BlackBladeInc.com twitter.com/BlackBladeInc www.blackbladeinc.com/pages/blog.aspx [email protected]


About this Presentation Intended audience

Application architects (Visio, not Visual StudioSome distributed application design

experience is helpful Not a session on writing the ultimate

web part Focused on HOW the application

components are organized, not WHAT the application does


About You How do you use SharePoint today? Why do you want to learn about using

SOA to create SharePoint apps? If you could make just one problem go

away with the snap of your fingers, what would it be?


Overview Problems with Traditional SharePoint

Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager

application to docBlock service


Problems with Traditional Problems with Traditional SharePoint Application SharePoint Application ArchitectureArchitecture


Sample SharePoint Application

Sha

reP

oint

Far

m

Application Data Storage

SQL Server

Application Interface

Web Front End

Application Logic

Data

Data

Application Page

Web Part

Application Logic


Aspects of the Architecture Entire codebase must reside on each

web front end server Web front end servers bear the bulk of

the load for executing the code Most code runs in the w3wp.exe

process Most code runs in the context of an

HTTP request


Problems with the Architecture These are the “bad practices” of

traditional SharePoint application design Entire codebase must reside on each

web front end serverDeployment complexityConfiguration complexityPotential application compatibility issuesOne more thing to worry about when

upgrading to next version of SharePoint


Problems with the Architecture Web front end servers bear the bulk of

the load for executing the codeDifficult to determine load application

places on farmMay require scaling out the web front ends


Problems with the Architecture Most code runs in the w3wp.exe

processRecycling the w3wp.exe process causes all

applications to restart


Problems with the Architecture Most code runs in the context of an

HTTP requestHTTP has a 60 second timeoutBad code can break the HTTP request

pipeline


Problems with the Architecture Difficult to copy protect application

Many files must be un-protected:○ Site and list definitions and templates○ Content types and other XML files○ Branding customizations (images, CSS, etc)


Problems with the Architecture Difficult to apply licensing

No extensible license management in SharePoint framework

Cannot easily deploy custom license management components to SharePoint


Problems with the Architecture Summary

Main issue is that all of the application’s components are being deployed to the customer’s SharePoint farm

The application has no control over the platform on which it is running

The fixSeparate the application from the farmLet the farm access the application as a

service


SOA IntroSOA Intro


SOA Intro SOA: Services Oriented Architecture SOA is architecture

Not a technologyNot a product


SOA Myths Myth: SOA is new Myth: SOA needs an Enterprise Service Bus Myth: SOA requires a particular technology

Myth: SOA requires Web ServicesMyth: SOA requires XMLMyth: SOA requires text-based interfaceMyth: SOA requires HTTP

Proof: Look at WCF in .Net 3.0


What’s Left in SOA Definition? SOA is a design principle that abstracts

a business capability from the interface used to access the capability.

SOA services should:Scale upScale outBe distributable

○ Have remotable interfaces○ Be topology-aware


SOA Service, not Web ServiceS

OA

Ser

vice

SOA Service Data StorageSOA Service LogicSOA Service Interfaces

Data

Data

Data

HTML

Web Service

RSS

XML

SMTP

Service Logic

DCOM

Terminal

CORBA

Interface Business Capability


What about the Clients? The User is not the client HTML Clients: IE, Firefox, Safari RSS Clients: Outlook, NewsGator SMTP Clients: Outlook, Notes, Evolution Terminal Client: HyperTerminal, x3270 Web Service Client: ? – Custom Code XML Client: ? – Custom Code


Who cares about the Clients? Users

The service client is the users’ only interaction with the service, so make the service client shine.

Sales peopleIt’s very difficult to sell a back-end service.

Much easier to sell the service client. Marketing people

You can’t make brochures or screencasts of a service. You can of a service client.


Rethink the Application as Rethink the Application as ServicesServices


Design Goals Increased stability

Customers don’t like applications that periodically stop working

Increased scalabilityIf your product doesn’t scale, your customer

has no reason to buy more than 1 Increase likelihood of getting paid for

your product


What Impedes Stability? Platform updates and configuration

changes Platform variations

Windows Server 2003 / R2 / 2008 / R2 / 2012

WSS V3 / V4 / V5 SharePoint Server 2007 / 2010 / 2013x86 / x64

Permissions changes


What Impedes Scalability? Poor application architecture


What Impedes Getting Paid? Poor adoption Piracy (intentional and unintentional) Unlicensed users, servers, site

collections No upgrade purchased because original

product was not usedDifficult to deployStability problems


Achieving the Design Goals Re-architect the application as a SOA

Service Get as much code off of the SharePoint

farm and out of W3WP process as possible Secure, copy-protect, and license the

service Spend as much or more effort on the

service client as on the service Distribute the service client freely


Our Service Client The SharePoint farm is the service client

SO

A S

erv

ice

SOA Service Data StorageSOA Service LogicSOA Service Interfaces

Data

Data

Data

HTML

Web Service

RSS

XML

SMTP

Service Logic

DCOM

Terminal

CORBA

Interface Business Capability

Sha

reP

oint

Far

m

Application Data Storage

SQL Server

Application Interface

Web Front End

Application Logic

Data

Data

Application Page

Web Part

Application Logic


Achieving the Design Goals Separate processing into 2 categories:

Requires SharePoint infrastructure Does not require SharePoint infrastructure

Separate processing duration into 3 categoriesFast (< 1 sec)Medium ( > 1 sec, < 60 sec) long running ( > 60 sec)


Design Implementation


My code fits more than one box Of course it does. It should. Most non-trivial applications will use at

least 2-3 boxes.


For Example… Box 1: A click in a web part causes a

document property in SharePoint to change. Box 2: The property change triggers an

asynchronous event receiver to call a remote web service.

Box 4: The remote web service starts a long-running operation.

Box 6: The long-running operation completes and publishes the results back to SharePoint.

Box 1: The web part displays the new state of the data when a user revisits the page.


Re-architecting an Application Process flows are the key to re-

architecture Reuse conceptual processes, not code Create good remotable service APIs for

the service client to invoke


Case Study: Re-Case Study: Re-architecting the Virtual architecting the Virtual Document ServiceDocument Service


Document Section Manager App What it did

Software that allowed multiple people to edit the same document at the same time

How it did itConverted a physical document into a

virtual document with multiple physical sections

Merged section changes back into main document so main document was always current


Document Section Manager App Implementation

Document section changes trigger an asynchronous event receiver

Event receiver invokes a custom Windows service to process document sections

Windows service invokes Merger Engine through DCOM to perform the section merges

Another Windows service uploads the processed document back to SharePoint


DSM Process FlowD

ocum

ent S

ectio

n M

anag

er P

roce

ssin

g

Assembler Merger Engine UploaderEvent

Receiver

Section Change Detected

Processing Request Received

Merger Engine Initialized

Engine Merges Sections

Assembler service passes merge

results to uploader service

Uploader service uploads merged

document to SharePoint

Updated Virtual Document in SharePoint

Virtual Document is “Processing”


DSM’s Place in SharePoint Farm

Redundant Failover HTTP

Load Balancers

Document Section Manager

Load Balanced SharePoint 2007 Web

Servers

Clustered SQL 2005 Database

Back End Servers

SharePoint Users

SharePoint Index and

Application Servers

Merger Engine

Event Receiver

Document Assembler

Document Uploader

Management Pages

Content Types

List Definitions


Issues with the Architecture All processing ran on each web front

end Deployment was a nightmare

Deploy and configure Merge Engine to each WFE

Deploy and configure custom Windows service to each WFE

System reliability was sub-optimal, mostly due to constant DCOM issues


Issues with the Architecture Application did not work when

SharePoint was installed on Windows Server 2008, due to DCOM differences

Application did not work when SharePoint was installed on Windows Server 2003 x64

Document processing put substantial load on WFEs


Issues with the Architecture Main issue:

The Document Section Manager was too intrusive in the SharePoint system. This caused the deployment, configuration, stability, and maintenance issues.

This is an architectural issue that required a fundamental redesign of the application.


Re-architecting the DSMD

ocum

ent S

ectio

n M

anag

er P

roce

ssin

g

AssemblerService

Merger EngineUploaderService

Event Receiver









Updated Virtual Document in SharePoint


1 3 4 & 6

3 4 & 6

3 6


docBlock’s Place in SharePoint

Redundant Failover HTTP

Load Balancers

Load Balanced SharePoint 2007

Web Servers

Clustered SQL 2005 Database

Back End Servers

SharePoint Users

SharePoint Index and

Application Servers

Service Client

Event Receiver

Management Pages

Content Types

List Definitions

Service Interface

Merger Engine

Document Assembler

Document Uploader

Virtual Document Processing Service

15% 85%


Re-architecting the DSM Document Assembler Service remains

mostly unchanged but is moved from SharePoint farm to docBlock server appliance.

Uploader Service remains mostly unchanged but is moved from SharePoint farm to docBlock server appliance.


Re-architecting the DSM New SOAP Service interface created for

Document Assembler and Uploader services on docBlock server appliance.

Event receiver remains mostly unchanged but now invokes the Document Assembler Service remotely rather than locally.

Merger Engine is removed from SharePoint WFEs.


docBlock Client / Service Flow

docB

lock

Ser

vice

Pro

cess

ing

AssemblerService

Merger EngineUploaderService

docBlock Web Service Interface









Web Service Returns

Processing Complete

docB

lock

Clie

nt P

roce

ssin

g

Event Receiver



Data is prepared for docBlock

service invocation

docBlock service invoked

ServiceClient


Tech Benefits of Re-architecture Deployment is 80% easier 85% less code running on SharePoint

WFEs 95% less load on SharePoint WFEs 75% increase in processing reliability


Tech Benefits of Re-architecture Support for SharePoint on all Windows

Server versions, including x86 and x64 Support for all versions of SharePoint,

including 2013 Cloud-ready architecture – important for

Office 365


Non-Tech Benefits Easier to find developers Easier to sell Companies willing to pay us to help

them similarly re-architect their applications.


Non-Tech Benefits Application can now be distributed as a

virtual or physical appliance. Application can now be hosted “in the

cloud.”


ClosingClosing


Summary If processing doesn’t require SharePoint,

don’t run the processing in SharePoint. Minimal intrusion into SharePoint yields

a more stable, reliable, performant platform.

Utilizing SOA service interfaces yields more manageable, upgradable, and supportable code.

The SOA service is “the product”.


About Black BladeSpecialize in SharePoint document and records management, information architecture, distributed systems, and systems integration

www.BlackBladeInc.com twitter.com/BlackBladeInc www.blackbladeinc.com/pages/blog.aspx [email protected]

using service-oriented methodologies to create sharepoint applications by eugene rosenfeld -...

Documents