Nov. 2016#18

®SharePoint

Office 365 collaboration tools: What to use for what today?

Visual Studio Team Services Release Management for O365 product development

Azure Information Protection

Implementing SharePoint as a collaboration site

Introducing the PnP Partner Pack v. 2.0

Microservices: The good, the bad and the ugly

Thanking all sponsors

and authors for the

past 7 years DIWUG is

looking forward for

your cooperation

during the years to come.

eMagazine #18 - November 2016

2

ContentsOffice 365 collaboration tools: What to use for what today? 4

Visual Studio Team Services Release Management for O365 product development 11

Azure Information Protection 19

Implementing SharePoint as a collaboration site 29

Introducing the PnP Partner Pack v. 2.0 33

Microservices: The good, the bad and the ugly 38

About the Authors 46

ColofonDIWUG SharePoint eMagazineNr. 18, November 2016

Publisher:Stichting Dutch Information Worker User Group (DIWUG)http://www.diwug.nl

Editors:Marianne van Wanrooijmarianne@diwug.nlMirjam van Olstmirjam@diwug.nlMaarten Eekelsmaarten@diwug.nl

Special thanks to:All authors and sponsors!

Design and layout:Barth Sluytersbarthsluyters@telfort.nl

©2016. All rights reserved. No part of this magazine may be reproduced in any way without prior written permission of DIWUG or the author. All trademarks used in this magazine are the property of their respective owners.

Possible copyrights on images and/or text supplied by the authors are for the sole responsibility of the author concerned. In no event can neither DIWUG, the producer, nor the printer of this magazine be charged for copyrights.

eMagazine #18 - November 2016

3

This issue is sponsored byAvanade: Dream. Big. 18

When using the Adobe® Acrobat® version all sponsor information is hyper linked.

Editors noteMagazine number 18! And this magazine is somewhat special. In this magazine you will find only one advertisement. It’s from Avanade, who has been our sponsor from the beginning of the printed version of this magazine as they we’re so kind to cover the printing costs. Together with other sponsors we we’re able to design, publish and deliver this magazine twice – three times a year.

For this edition DIWUG is paying for the costs for publishing this edition. Of course, this is something we can’t do too many times. Please contact us if you or your company is interested in sponsoring this magazine and we will send you more information on pricing and publishing details. We would like to continue! But we can’t without the sponsors and of course the authors!

And we have again a great line up of articles. We always try to get a good mix of end user/business, developer, architectural/IT Pro articles. In this magazine we have the following articles for you lined up:

End user/businessMaybe you’ve read the “What to use for what”-article of Frédérique Harmsze in magazine number 17. A lot has changed since that article was published! That’s why Frédérique really wanted to write an update. This is a must read for everyone who’s working with and consulting on the collaboration tools of Office 365.

Robert Wetting and Ronald Groeneweg has written a two part article on Implementing SharePoint as a collaboration site. In this magazine you can read part 1. It’s an article written from a business point of view on how the implementation process could (or should) look like.

DeveloperIf you are a developer, you really want to read the article of Clemens Reijnen on Visual Studio Team Services Release Management. In his article he leads you step by step on setting up the VSTS release management with practical tips and links.

Introducing the PnP Partner Pack by Paolo Pialorsi gives you a better understanding on what to expect of the Developer Patterns and Practices for SharePoint/Office365. PnP is the open source solution for the most common tasks of managing your sites and site collections (e.g. remote site provisioning).

ArchitecturalAzure Information Protection is a new cloud service in the Azure stack. Together with Azure Rights Management you’ll get a very powerful way to protect you documents even when you have shared them with others (even external). Albert Hoithing takes you through his journey of discovering this new and awesome feature!

You might not think of Microservices when building applications for SharePoint or Office365. It’s an interesting concept on architecting and deploying applications. Microservices is a way to design your applications into small, independently deployable services. Microservices makes for example the automated deployments (as described in the aticle of Clemens Reijen) easier. More on Microservices you can read on http://martinfowler.com/articles/microservices.html. Although the concept sounds interesting, there are a couple of things to consider. Sander Hoogendoorn tells you all about the Good, The Bad and the Ugly.

Enjoy!

eMagazine #18 - November 2016

4

QUOTE

OneDrive for Business now is a starting point for collaboration

Office 365 collaboration tools: What to use for what today?

by Frédérique Harmsze

The Office 365 toolkit is evolving rapidly. The Office 365 Groups are integrating with SharePoint Sites, Yammer will soon be integrated seriously too, and OneDrive for Business offers a relevant

collaboration scenario. So which tool can we use for what today?

In the previous edition of DIWUG Magazine (number 17), we discussed the collaboration tools and what we could use for what. Soon after the magazine was sent to the printers, things had changed. The basic recommendations were still valid, but some of the limitations I complained about had already been resolved. Since then, even more improvements have been implemented.

This article is a re-evaluation. Let’s determine what I think should be used for what today. Of course it may well be different tomorrow, but that’s how it works with the very dynamic Office 365 toolkit.

OneDrive for Business: Your digital desk drawerMy personal storage space within Office 365 is OneDrive for Business. The files live in a limited version of a modern SharePoint library: it looks modern, but it lacks the options to create interesting views.

New is the option to copy files from OneDrive for Business to another Site in Office 365. It appeared in my tenant literally today. The option to move the file, instead of copying, will follow soon. This means that my files are no longer stuck in my personal digital desk drawer: OneDrive for Business now is a starting point for collaboration.

Figure 1: The new option in OneDrive for Business to copy a file to a SharePoint site.

Use OneDrive for Business for: Working on a first draft.

With the new Copy button, it becomes easy to start a draft in your OneDrive for Business, share it with a colleague to achieve a presentable version, and then copy (later: move) it to a SharePoint Site for serious collaboration with the rest of the team.

5

Continuing work on documents stored anywhere in Office 365

OneDrive for Business also offers an overview of documents that you may want to work on now: the documents that you have modified recently or that are shared specifically with you anywhere in Office 365. The new Delve-based Discover view makes OneDrive for Business even more of a “personal document portal”.

Ad hoc sharing with colleagues or external contacts.

This is still valid. For example, you can share a big presentation with a client by storing the file in OneDrive for Business and e-mailing a link (provided external sharing is enabled in your tenant). But if it is part of a project, share it in the project Site.

Storing work-related documents that are relevant only for you.

This is still valid. You can put files about your personal development, for example, in here.

Do not use OneDrive for Business for: Storing content that is important for the organization.

This is still valid. OneDrive has no Enterprise Content Management options. And most of all: you are the only owner of the files. When you leave the organization, your manager has a month to grab what might still be relevant, and then your OneDrive for Business is deleted and the files may be lost.

Systematic collaboration, over a longer period of time, with more people.

This is still valid. Starting collaboration in OneDrive for Business is fine, but as soon as the draft becomes relevant to your team, it should move to a Site where it is managed more seriously.

A quick note on storing a file in OneDrive for Business and sending your contact a link to it. Outlook on the web offers the option to upload your attachment from your computer into OneDrive for Business, instead of attaching a copy of the file to the message in the classic way. The Outlook client does not offer this upload option. Both Outlook on the web and Outlook 2016 do allow you to select a file that is already stored in OneDrive for Business and attach it as a link. So you are better off if you put your document in OneDrive for Business instead of in your old fashioned My Documents on your computer.

Figure 2: In Outlook on the web you can upload a file from your computer into OneDrive and send the attachment as a link

6

QUOTE

Office 365 have matured a lot

Office 365 Groups: Our digital officeThe Office 365 Groups have matured a lot since the previous article. The set of Office 365 services for collaboration that the Office 365 Groups bring together is more complete, and the different services are connected more smoothly. For example, Planner is turning up in the menu.

The key improvement, which has been rolled out to my tenants a few days ago, is that new Groups now have a full SharePoint Site, instead of just a truncated Document Library. So in most cases you don’t have to choose between an Office 365 Group and a SharePoint Site. You get both when you create a Group.

Now we can add pages and lists in the SharePoint section of the Office 365 Group, set up the homepage to show what is most important and create different views to allow the user to find the relevant information from different angles. We can now also facilitate processes with workflows, using the new Flow or Nintex Workflow for example. And define Content Types and Site Columns for systematic content management.

Figure 3: The Files section in a Group, which is now part of a full SharePoint Site

The SharePoint Sites that are associated to my new Groups, however, are not entirely complete: the Site Permissions options are missing, as are the Audit Log Reports and design related options. Mark Kashman (Senior Program Manager for SharePoint Online and Office Delve at Microsoft) stated that the Site in the Group does have everything, but the features that would confuse non-SharePoint users are hidden. User management is supposed to take place via the Group settings in Outlook; unfortunately, that is less powerful. And the audit logs live in the Compliance Center, because the Group also has email, but that is less accessible.

External access is another key improvement. Office 365 Groups now allow you to collaborate with people outside your organization. They can join the conversation via Outlook and they are Members of the SharePoint Site associated to the Group; so they even have permission to manage lists. However, guests lack the other options in the Group: they don’t see older conversations, the calendar, tasks in Planner or the members of the Group.

Use Office 365 Groups for: Collaboration in general.

It only takes minutes to create a new Office 365 Group using the self-service process. And then you can share with your colleagues: conversations, a calendar, files, notes, pages, lists, and tasks in Planner. This is a step up from last time, when I could only recommend Groups for basic or temporary collaboration, because the truncated Files section was too basic.

7

QUOTE

Did somebody say that SharePoint was dead?

It is more alive and widespread in Office 365 than ever.

Collaboration with external people.

As soon as I could invite external contacts, I created an Office 365 Group Site for a small project, to share files with the client who was emailing me documents. Actually, apart from the conversation that takes place via the Group mail address, this is SharePoint. But at the moment, creating a Group is easier and faster than creating a SharePoint Site directly.

Collaboration with Outlook devotees.

This is still valid. Office 365 Groups are a great way to collaborate with people who adore Outlook, especially if that is Outlook 2016. The Planner button is still missing there, but the Calendar, Files, Notebook en Members are available from the client.

Do not use Office 365 Groups for: Highly regulated content, unless you are sure of the required safeguards.

This is still somewhat valid: governance on Office 365 Groups is getting more serious, with for example data classification for policies and visibility in the Admin portal. But not as serious as in a native SharePoint Site. So if you have to prove to some authority that you did everything by the book, do not store the information in an Office 365 Group until you are sure that it can meet that authority’s requirements.

Facilitating advanced processes, unless you are sure your requirements are met

Now that we can create our own lists with workflows in the SharePoint section of the Office 365 Group and Planner is becoming visible in the Group, we can use Office 365 Groups to support basic processes. However, I am hesitant to put complex processes in there: I often use SharePoint groups to assign tasks in Nintex Workflows, and we cannot do that now.

Publishing information for many readers.

Office 365 Groups are for collaboration and not for publication. We cannot simply invite people as Visitors, only as Members who can edit. Rather than trying to make that work, just use a native SharePoint Site: take the hammer to hit the nail, instead of trying the screwdriver…

For now: un-savvy users if they don’t get guidance.

The functionality offered by Office 365 Groups is great. But the usability of the navigation is still not so great. When you are in the associated SharePoint Site, it is unclear how you get to the Conversations and Calendar that live in Outlook (hint: click on the Site title for the menu). And how do you get out of the Calendar? Savvy users will find their way, but you will lose innocent users who are reluctant to embrace new tools, unless you explain it carefully.

SharePoint Sites: Our digital workshopDid somebody say that SharePoint was dead? It is more alive and widespread in Office 365 than ever. Even the name SharePoint is back in Office 365; we no longer just talk about “Sites”.

New Office 365 Group automatically have an associated SharePoint Site, and you can still create “native” SharePoint Sites. At the moment, these native SharePoint Sites do not have an associated Office 365 Group, but that will be implemented soon. The lists below focus on native SharePoint Sites as we see them now, before Sites and groups are fully merged.

8

Use SharePoint Sites for: Facilitating advanced processes.

Advanced processes facilitated with workflows may move across different tools. But they are usually based in a SharePoint Site, which hosts the application. It could also work in the SharePoint Site associated to a Group, but relevant features may be hidden there, at least for now.

Highly regulated content.

The advanced Enterprise Content Management options are all visible in SharePoint Sites, like fine-grained permissions to determine exactly who can do what (for example, the group of compliance managers can edit but not delete) and audit logs to prove who did what.

Making information available to large groups

SharePoint Sites are not necessarily Team Sites for collaboration. You can also create Publishing Sites. Usually that template is most suited for publishing information to many readers.

Do not use SharePoint Sites for: For now: Quick collaboration.

It takes more time to set up a SharePoint Site than an Office 365 Group. So until the new self-service Site creation tool becomes available, it won’t be worth the effort to create a SharePoint Site if you want to collaborate quickly.

Personal documents that are only relevant for you.

This is still valid: Put files irrelevant for the organization in OneDrive for Business.

Conversations.

This is still valid: SharePoint itself only has very basic Discussion List template. The Conversations in Office 365 Groups and Yammer Groups are a lot more practical. The modern Site Page has an easy way to embed Yammer, to display the conversation in the context of SharePoint.

Figure 4: The homepage of a modern Team Site associated to a public group

Yammer: Our digital water coolerYammer is still the place where we can hang out, grab a drink and discuss what’s on our minds – except for the drink. Microsoft is planning deeper integration with the rest of Office 365. Yammer Groups will become similar to the Office 365 Groups we have now, with the same association to a SharePoint Site and Planner, but with the conversation in Yammer instead of in Outlook. Then you can pick the group type that suits your preferences. But for now, it is still an Enterprise Social Network focused on conversations.

9

QUOTE

Yammer: hang out and discuss what’s on our minds

Missed the last DIWUG event?UnityConnect 2016 - Haarlem

16th november 2016

Extending PowerApps and Microsoft Flow with custom code - Adis Jugo, skybow

Creating a Great User Experience in SharePoint - Marc D Anderson, Sympraxis Consulting

Don’t let it happen again. Go to http://www.diwug.nl for the oncoming event.

I have not seen any major changes in Yammer since the previous article, so this is a quick summary:

Use Yammer for: Reaching out to the entire organization

Post a formal announcement or an informal message in the All Company group, so that everybody can read it and respond.

Discussions and questions in your organization.

If you have a question and you are not sure who can answer it, post it in Yammer, preferably in specialized Yammer groups.

Discussing videos in the Video Portal

In the Office 365 Video Portal and Delve, you have the option to discuss the items in Yammer while the files stay where they are. You can see the discussion in Yammer, as well as in the context of the item. However, it does not work as nicely for documents now. The option to post to Yammer is available in classic Document Libraries as well, but the discussion is not displayed in the context of the document. And the option is missing in modern Document Libraries.

Discussions and questions with externals

Yammer offers external networks and external groups. If the conversation with these external contacts is important and collaboration on documents is not, then Yammer is a good option.

Do not use Yammer for: Systematic collaboration involving documents or processes.

Currently, a document in Yammer is an attachment to the conversation that you cannot manage or find properly. And there is no user friendly way of discussing a file stored in a modern Document Library. So I’ll wait for that deeper integration.

Posting long stories

Yammer post are hard to read, because you don’t have the option to format text.

10

QUOTE

The different tools are integrating better

Figure 5: Starting a Yammer discussion from a document found in Delve.

ConclusionThe toolkit of Office 365 is evolving rapidly. More tools are added and the tools get more powerful. And most of all: the different tools are integrating better. This means you don’t have to make an irrevocable choice as to where and how you collaborate.

You move from one tool to the other seamlessly. For example: start writing a document in OneDrive for Business, send a link to it as an attachment in Outlook asking a colleague to help, talk it over with her in a Skype for Business meeting by sharing your screen. Then copy the first draft to the SharePoint Site in an Office 365 Group for serious collaboration and discuss it in the Conversation section of the Group.

That works today. How about tomorrow? Let’s keep an eye on it and find out.

References Office 365 collaboration tools: what to use for what? Frédérique Harmsze, DIWUG Magazine #17, June 2016

http://roadmap.office.com

https://blogs.office.com/2016/09/26/yammer-strengthens-team-collaboration-through-integration-with-office-365-groups/

https://blogs.office.com/2016/10/12/office-365-groups-update-at-ignite-2016/

eMagazine #18 - November 2016

11

TIP

A 101 on setting up a VSTS with Git and GitFlow can be found on this post.

(http://www.clemensreijnen.nl/post/2016/07/11/101-Setup-configure-and-work-with-GIT-VSTS-Build-and-Release)

Visual Studio Team Services Release Management for O365 product development

by Clemens Reijnen

The scenarioA team is working on a Provider Hosted SharePoint Add-in, hosted on Azure with multiple deployments for different O365 tenants. The development team wants a friction free, controlled and with as less as possible manual tasks delivery of the application to the different customers. They want an automated delivery pipeline, Continues Integration and Continues deployment or CI/CD for their SharePoint Add-in.

Having a robust delivery pipeline for your application that is still under development has many benefits. An important benefit can be that everybody in the team can start a release of a new version of the application. Which makes it possible to release more often. Another benefit is a higher quality of the functionality of the application. The release and automated testing is done every time in the same way, which makes it much more reliable.

Every application is different and requires other ways and needs for the configuration of the CI/CD pipeline. An extension to a SaaS system (O365 Add-in) can be challenging due to the dependencies on that application.

In this article the steps a team need to take to setup a CI/CD pipeline in Visual Studio Team Services for add-in for Office365 are explained.

The first setupThe team uses GIT for versioning artifacts together with Visual Studio Team Services (VSTS) to organize their sources, backlog, tasks, builds and releases. A default GitFlow policy is used for branching the structure with a drop to GitHub for the created Open Source components. Azure Resource Manager JSON definitions are used for creating the Azure Resources used by the application.

Versioning everything is a good common practice for development teams. For a proper CI/CD pipeline first a versioning strategy needs to be decided on. GIT is, at this moment, the main choice for teams as their versioning system, together with VSTS and the capability to configure branch policies, makes it a stable and good controlled way of accepting changes for new release.

The Azure Resource Manager (ARM) is a JSON definition description of the created and used Azure resources for the application. Simply the description of the infrastructure is needed to run the application. In the context of the CI/CD practice ‘versioning everything’ and ‘automate all tasks’ are ARM definitions a valuable artifact for teams. An ARM definition can be put under version control and they can be used to create all the needed Azure resources and corresponding configuration via PowerShell. ARM makes it possible to follow these DevOps practices.

12

QUOTE

Git really changed the way developers think of merging and branching. How to create a deployment model?

Go to: http://nvie.com/posts/a-successful-git-branching-model/

QUOTE

Branch policies help teams protect their important branches! https://www.visualstudio.com/en-us/docs/git/branch-policies

TIP #1

Keep the value of the AppManifest ClientId to a wildcard ‘*’. Changing this will break all other developers debugging experience.

TIP #2

Team members often forget this one, when the SharePoint App hasn’t changed, disable it as a startup project when debugging. The time consuming ‘retracting SharePoint app – installing SharePoint app’ activities are skipped and you can validate the application much faster. You only have to run it with the SharePoint app enabled as startup project once to get the ‘on the fly generated’ ClientId in the app. From then just debug the WebApp only.

Figure 1: VSTS/GIT Hub Deployment set up architecture

The Development teamEvery team member uses its own developer instance of O365. That is… maybe some have Visual Studio MSDN Professional which doesn’t come with a O365 developer tenant. They are working on developer Sites Collections created on the company O365 tenant.

For more information, go to: https://www.visualstudio.com/en-us/products/compare-visual-studio-2015-products-vs.aspx.

A normal GitFlow is used by team members with branch policies on the develop and master branch for pull requests, linked work items and code reviews.

Two developer tips:

13

TIP

Very important. When a project also commits to a public repository, or when you work on a public Git Hub repository remove all Azure keys and other security information (connectionstrings) from your sources before pushing it to GitHub. Read this post why: https://www.humankode.com/security/how-a-bug-in-visual-studio-2015-exposed-my-source-code-on-github-and-cost-me-6500-in-a-few-hours

VSTS + Zapier + GitHubSources for this application in this scenario are also pushed to GitHub when a commit is done on the master branch via Zapier.

For example the Sogeti Site-Provisioning app is committed to GitHub via Zapier (see: http://clemensreijnen.nl/post/2015/10/29/Sogeti-O365-Site-Provisioning-(on-GitHub))

Figure 2: Push sources from VSTS to GitHub

Azure Key Vault for Application settingsTo be sure Azure keys, and other application settings, aren’t used for the above mentioned horror scenario the Azure Key Vault is used by the application. The Azure Key Vault is connected to Azure Active Directory and only applications and users registered in the Azure Active Directory are able to access the keys.

https://azure.microsoft.com/en-us/services/key-vault/

Happy secure, but still the build removes also these keys from the config files, the real keys are next to the web application in the Azure Web App settings pane as shown in figure 3.

Figure 3: Azure Key Vault

14

“/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true”

The Build.Multiple builds are taking care of the code. Builds are triggered continuously on the develop and master branch when a pull request is approved. These builds compile, validate and package the WebApp and other ‘non-SharePoint’ components. One additional build is responsible for building and making the SharePoint app packages. A smaller and faster build validates the pull request on integration issues.

Figure 4: The Build

The WebApp build has several build steps. Two of the four steps are Visual Studio tasks, a Visual Studio Build with the build arguments for building and packaging the Azure Web App solutions (Listing 1), and a Visual Studio Test build step for running the unit tests(figure 5). So you get the nice looking informative test results.

Listing 1: Build arguments for building and packaging the Azure Web App solutions

Figure 5: Build and test results

There is one additional PowerShell build step for the Azure Web Job packaging, this Azure Web Job is part of the application for running UI-less recurrent tasks. For some reason the default package method of Visual Studio MSBuild creates a zip file for the Azure Web Job with an enormous directory hierarchy. This directory hierarchy makes the deployment fail (see release paragraph). The PowerShell build step simply takes the Web Job compile output and puts it in a zip.

Figure 6: PowerShell listing: Put the WebJob compile output in a simple zip

The next build step is the Copy Files that copies the zip files and the PowerShell files to the drop folder on the server. These are needed for the release process.

Before releasing, during the build the code is validated via SonarQube for quality. SonarQube software (previously called Sonar) is an open source quality management platform, dedicated to continuously analyze and measure technical quality, from

15

project portfolio to method. In this scenario a quality gate is set for all code on “smells” , SonarQube validates these smells and will throw a failed message when the new code doesn’t comply with the rules.

Figure 7: Quality check by SonarQube

See also: http://www.codewrecks.com/blog/index.php/2015/06/30/manage-artifacts-with-tfs-build-vnext/

The Create SharePoint Apps build.The application we are building and deploying in this scenario needs multiple SharePoint Apps, per customer installment and per stage a SharePoint App. As you can see on figure 8, every stages got its own SharePoint app with the same features, but with different security settings.

The deltas per SharePoint App are the ClientId, ClientSecret and the URL to the Web Application. All other differences in the SharePoint App settings are organized via feature flag settings.

Side note: it is easier to make a multitenant solution, register the SharePoint App in the Office Seller dashboard and organize the customer specific settings via feature flags. You only need to register one SharePoint App per stage.

Figure 8: Every SharePoint App gets its own properties

16

“/p:ActivePublishProfile=”ProfileToBuildAppFor”

Creating a SharePoint App packages via Visual Studio is easy. Right click on the SharePoint project and select publish. In the dialog select the Publishing profile and move on. For creating multiple SharePoint Apps via VSTS Build the Publishing Profiles are the key items. Create for every needed configuration (ClientId, ClientSecret and the URL to the Web Application) a publish profile. Easy done via Visual Studio as shown in figure 9.

Figuur 9: Managing publishing profiles in VSTS

Via a MSBuild Build step where the SharePoint project is selected and with an MSBuild argument as shown in listing 2, the apps are created and ready to put in the AppCatalog.

Listing 2: MSBuild arguments for publishing into the AppCatalog

Putting the apps in the corresponding AppCatalog’s and update the sites which are using the App is still an annoying manual step.

The Release.The release process contains Development, Test, Staging and Production environments. Where the development environment is better to be named Development Integration environment, the place where the integration of all developer artifacts are validated.

Resource groups and subscription organization.The Development and Test environments are put in a different Azure Resource Group than the Staging and Production environments, this to keep a clean separation of costs and access right. Via Azure Active Directory and Resource Group RBAC the development team only has access to the Development and Test environments. Customer system installments live in different Azure subscriptions with Staging and Production in the same subscription and resource group (for swapping).

Release management environments Per customer a release with environments is configured in VSTS Release management. There is one release with an ‘empty’ environment, this one is used to validate the release of a complete new deployment. Through the release task ‘create or update Resource Group’ a complete new resource group with Azure Resources for the application is created as shown in figure 10. It will be deleted after the deployment of the bits and the test execution.

Figure 10: ARM Setup environments: Azure Group Deployment

17

From up to

“New-AzureWebsiteJob -Name $webSiteName -JobName $jobName -JobType $jobType -Slot $slot -JobFile $packageFile”

“Switch-AzureWebsiteSlot -Name $WebSiteName -Slot1 $SlotNameFrom -Slot2 $SlotNameTo -F”

Still available on

http://www.diwug.nl

This makes the team comfortable to rollout new environments in a repeatable way, and makes sure the Resource Group Template is up to date.

All other releases are ‘upgrade’ releases, the Azure resources are upgraded if necessary. Mainly the upgrade is about changes in the application settings (.config) of the Web App. At this moment the team uses PowerShell to update the settings. Although this should be possible via the release task ‘Update RG’, that one is on the investigation backlog.

Develop, Test and Staging have all three release tasks for deploying the application as shown in figure 11. These tasks are; deploy the Azure WebApp, deploy the Azure WebJob and run Visual Studio Test.

Figure 11: Deployment tasks

The WebJob deployment is an Azure PowerShell tasks with the command as shown in listing 3

Listing 4: Deployment from staging to production

All environments contain the run test Visual Studio Tests which contains not only the unit tests but also integration tests, validating the communication with the different Azure storage locations and SharePoint Online (O365), making sure that all settings are made correct. Which is still the most occurred reason why a release fails.

Listing 3: Azure PowerShell command for deploying the WebJob

It takes the zip file created during the build.

The Develop Branch CI build triggers only the internal release to the develop integration environment and test. While the Master Branch CI build triggers all releases. VSTS Release management at this moment doesn’t have the capability to link or run releases in parallel. To work around these manual approvals is to put before releasing to production the app to a staging environment. After the Develop build has gone through the internal release stages, the pre-approval is made for the customer releases to the customer staging environments.

The last release step, from staging to production, is a single PowerShell command as shown in listing 4. Which swaps staging with production.

Avanade: Dream. Big.

18

Figure 12: Visual Studio Tests and deployments

Conclusion: Happy comfortable releasingIt is a must for teams running projects to setup a release pipeline like this. Too often team members are code focused only, forgetting that an application also needs to run somewhere. “Shy” team members who are hiding themselves and code in a feature branch and don’t integrate their code via pull requests can look busy, but they don’t deliver any value for the project. At least as long it isn’t touched by a user.

Creating a CI/CD pipeline for everything your team delivers will make it easier to share your hard work to the world and make your work valuable. Sometimes it is hard, definitely when external systems (O365) and hard dependencies are involved. But, it really pays off in delivery speed and quality when the team invests in the common team practices, versioning and automate everything.

eMagazine #18 - November 2016

19

Azure Information Protectionby Albert Hoitingh

We live in a world where information is readily available and can be shared with a simply press of a button. Protecting intellectual property (especially unstructured information like documents and

emails) in this modern world is getting harder and harder for organizations.

Dutch post: The Dutch “State of the union” leaked

There are stories abound of employees simply copying files to an USB-stick or storing them on a personal web storage platform (Dropbox, OneDrive, you name them) or attaching documents to an email and mailing it to a personal account.

Or what about the details of the Dutch “State of the union” speech, which always are delivered on the 3rd Tuesday in September. These details are provided to members of parliament at the last minute, with strict orders not to disclose them. But not to worry, because it’s become a Dutch tradition that at least one week beforehand most of these details are published by Dutch journalists.

So, it should not come as any surprise that data leakage prevention and dealing with sensitive information is very important to many enterprises. Microsoft is investing heavily in technologies to support prevention and protection. Now it’s time to take a closer look at a new (and as yet preview) offering of Microsoft: Azure Information Protection.

What is AIP?Although technologies like SharePoint Information Rights Management and Rights Management Services have been around for some time, Azure Information Protection (or AIP for short) is a new offering in the Azure stack.

But it is not technology which makes AIP new or improved. It’s the way Microsoft tries to make information protection easy for both the user and the IT department. Most of the time users just want to be able to work on documents and share them easily. Some might be aware of the information which is stored in these documents. But many will probably not recognize the need to be protective, or forget about the company policy regarding sensitive data.

So, one of the first steps is to make the user aware about the data sensitivity within those documents. After this, we can protect and monitor the information and respond when necessary. These are the functions AIP provides.

20

A quick note

In order to ensure that documents are protected (encrypted) using AIP, AIP needs Azure Rights Management (RMS). AIP and RMS are complimentary to each other, but are still separate functions. Where AIP is used to label documents, RMS is designed to protect them by using encryption. This is not just to make them harder to open; it is also to provide content expiration (for example).

RMS has its own add-in (Share Protected), which works in Office, Windows Explorer and is also available as mobile app. I won’t cover this in here. The AIP add-in uses some of the functions of this RMS add-in, like the tracking of documents. I won’t go into these functions either.

Figure 1: Microsoft Protection Process

How does this work?

A person’s perspective

Sign-inOur user has Office 2016 with the AIP add-in installed. She opens Word to start on a new document. Using her user-account, she’s signed into Office 365 (Azure AD). If single sign-in has not been enabled, our user is prompted to sign-in. After sign-in, the AIP add-in will retrieve the relevant labels.

Figure 2: Sign-in to Azure Information Protection

21

Labeling informationBecause she’s working on a company internal document, she selects the label Internal. Based on this label, AIP will apply a policy to the document. For example, a document header is placed on the document and a rights management policy is applied.

Figure 3: AIP labels within Office

Change the labelSeeing the document header, our user decided to downgrade the document to the Public label. This is possible, but she will need to enter a reason to do so.

Figure 4: Change the label

In the end, our user decides to just remove the label all-together. Who bothers with information labeling anyway? See can do this by simply clicking the recycle-bin icon. The sensitivity level will be changed to “Not set”.

Unfortunately for her, the document cannot be saved without a label. This labeling has been set as mandatory. So, before saving, Office will prompt the user to choose a label.

Figure 5: Mandatory label

In another scenario, our user has accidently entered some Social Security numbers within an Excel spreadsheet. When working on the document, AIP detects this sensitive information. It prompts the user to change the label or (if configured), the document label is changed automatically.

22

Configuring AIPAzure Information Protection is a separate module within Azure. At the moment of writing, it was still in preview. In order to use it, you will need (any of) the following (subject to change):

An Office 365 E3, E4 or E5 subscription;

Azure AD directory;

Windows 7 (SP1), 8, 8.1, 10 (x86, x64);

Android 4.0.3 and up;

iOS 7.0 and up;

Windows phone 8.1, 10;

Windows 10 Mobile and Windows 8.1 RT;

Word, Excel, PowerPoint, and Outlook (Office Professional Plus 2010, 2013 SP1, 2016).

The Office 365 AIP subscription will be divided in two, with a Premium package offering automatic content classification.

When users sign-in to the tenant (Office 365) and have the add-in installed, they can use the labeling functions. The IT department can assign actions to these labels.

Together with Azure Active Directory – Rights Management (RMS) you can also apply additional rights management policies to the document. You can configure this within the label. More about that later.

Figure 6: Azure AIP and RMS schematic (AIP client = RMS client)

Creating a AIP policyYou can access AIP from the Microsoft Azure portal. As it’s still in preview, it will have the “preview” (or in my case “Voorbeeld”) notification. You select Azure Information Protection and the default labels will be shown.

23

Figure 7: Microsoft Azure portal

Don’t be alarmed if it takes a few seconds for the labels to show up. They will eventually, but it might take some time. You are shown the labels which are displayed in Office. Out of the box, Azure will provide you with several commonly used labels.

The title is displayed in combination with the selected label. You can edit the available labels and/or add you own labels. At the bottom level, you can configure a mandatory level (all documents need to have a label). You can also set a default label. When users change a label (from more sensitive to less sensitive), you can configure AIP to ask for a reason.

When you are done, you publish the policy. After this, it will become available to new documents or update existing documents.

Figure 8: AIP policy

24

Adding an AIP labelAn AIP policy is made up of multiple labels. Let’s take a look at the settings of these labels. When opening a label, you will be treated to a multitude of options. These options range from:

Generic: Status, name, description and color of the label.

Visual markings: Header, footer, watermark

Complex options: Associated RMS template, automatic detection of content

In addition, you can add a mention to your users and some notes for the IT management people in charge of the label.

Figure 9 & 10: AIP label details (not the RMS section)

When the user selects the label, the document will automatically be modified to reflect the settings of the label. The user just has to make sure he or she selects the right label. To make this even better, you can have the label automatically detected.

Automatic content detectionAutomatic content detection is part of the premium offering of AIP. AIP will detect the classification of the content based on predefined policies. Credit card information, for example. You can select your policy from a list or create your own. This is somewhat similar to the Data Loss Prevention policies in Office 365. However, the AIP policies are not as detailed (yet?).

25

A quick note

At the beginning of the article I described the Microsoft process of protecting information.

AIP offers the Labeling function, where RMS offers the Protect function.

AIP is therefore mostly concerned with awareness.

RMS is mostly concerned with actions.

Both work side-by-side, but are better-together. That’s why it’s important to look at Azure RMS.

Figure 12: A polite suggestion to change the label(This document is best labeled as Secret)

Azure Active Directory Rights Management

Figure 11: Predefined policies for automatic content detection

When active, AIP will automatically detect sensitive content. Based on the AIP label, the user will either be asked politely to change the label (see below). Or the label is changed automatically.

26

Azure Rights Management (RMS) is a separate offering within the Azure stack. It is part of Azure Active Directory and it’s also the module which is needed when you want to use SharePoint Information Rights Management. RMS has a dedicated client add-in (working both in Office and the Windows Explorer). AIP uses RMS to enforce policies to documents.

Figure 13: Azure RMS policies (from Azure AD)

AIP and RMS work better together. It’s no use to have AIP labels and no RMS policy attached to them. And therefore, you can add RMS policies to the AIP labels. In all, these are the options AIP and RMS will provide.

A RMS policy can be configured to work for certain users within the organization and you can configure what these users can or cannot do with documents. As these options are quite extensive, I’ve included them in this figure.

Figure 14: AIP and RMS options

27

What about SharePoint?In Office 365 you can configure SharePoint to use Azure RMS. This way, you control the way documents are used from SharePoint. This functionality has been around some time and is called Information Rights Management (or IRM).

After you configure SharePoint Online to use RMS, you get an extra option within you document libraries. This option lets you set the options for IRM. You can disallow you users to print documents stored within SharePoint or restrict storing document which do not support IRM.

Figure 15: SharePoint IRM

Although SharePoint requires Azure RMS to support IRM, the similarities end there. SharePoint IRM is aimed only at documents which are accessed or downloaded from SharePoint. You will not see any AIP or RMS options either within the library or the management console. I expect that Microsoft will bring these platforms in line.

Figure 16: SharePoint IRM, DLP, AIP and RMS together

28

Figure 17: AIP, RMS and SharePoint IRM together

Now what?Information security is not always at the top of mind with the user community. With Azure Information Protection, you can have awareness and compliancy in one. To your user, it’s as easy as clicking a button. The IT department can be outfitted with a large array of information protection features.

The AIP add-in provides a very easy and understandable way for users to classify information. When a RMS policy is attached, this awareness is also transformed into information protection actions. The AIP platform is still in preview and I believe Microsoft has some backlog items to solve.

For example: how will the AIP and RMS Share Protected clients work together? And will IT management be notified when users continually downgrade the labels or even remove them?

I also do expect Microsoft to put effort into:

Providing one (integrated) solution based on data loss prevention, RMS, AIP;

Hybrid solutions based on the PKI an enterprise already has (BYOK - bring you own key);

Having IRM/RMS on site-collection or tenant level instead of document library level;

Integration with the Security & compliance center (for instance for monitor users lessening the labels).

Also, and perhaps more important: an organization will need to possess a certain level of information management maturity. Implementing IRM, RMS, AIP (or any other acronym) is not something to do over the weekend.

eMagazine #18 - November 2016

29

Implementing SharePoint as a collaboration site(Part 1)

... not to forget the human touch!by Robert Wetting and Ronald Groeneweg

Robert Wetting and Ronald Groeneweg wrote in 2015 a joint plan for a customer for the creation of a collaboration site in SharePoint. The plan was received with enthusiasm and the site was

implemented. Shortly after the summer the first departments went live in a pleasant and simple site in which the archive functionality is automated.

In two parts we describe our vision and approach to achieve a (implemented) collaboration site that is both workable and recognizable to end users and manageable for the IT department.

In the first part we focus on implementation of the collaboration site. In the second part in the next magazine (#19) we’ll discuss the aspects of information architecture, sense of quality, assignment change and risks during the implementation of the collaboration site.

A brief introductionAs we enter a collaboration site, we think of a digital site which is:

Consistent with what people are used to at home: a site that should not be more difficult to use than Facebook.

Where document collaboration (document management) is the base: all kinds of other ways of working together (‘social’) are also possible but not required to use.

Where digital archiving (records management) is automated: when an employee drops his document in the right file and adds the right features the document is automatically archived.

Ease of use, standardization, scope and speed are key words.

Ease of use means we work in a simple site without high barriers or dark corners. We comprehend what we do.

Standardization means we know a few work methods (process-oriented, project-based, open collaboration) and provide standardized support. We also do not allow software customization. We only apply customization during implementation (not every business is identical or has the same challenges).

Scope means we introduce a new way of working that is focused on the future. We try to minimize the miseries of the past (e.g. through migration.) Only when the new process is implemented do we no longer need to worry about spilt milk ‘and what to do with all that old information. The collaboration site does not undo thirty years of misery network drives, mailboxes and custom document systems.

Speed means we implement digital work as quick and simple as possible for the entire organization. If we take too long, the natural opposing forces have the chance to become stronger ... and thusly endanger the implementation. An organization is in fact just like a human being and thus rarely open to change.

30

The aim of the cooperation areaTo briefly summarize the goal of the collaboration site: “realization of a single collaboration platform which can be worked with ease both internally as well as externally and where at the same time digital archiving is automated”.

The collaboration site puts an end to “the fragmentation of information”. It fully and properly accesses all relevant digital information and creates a wealth of new opportunities for modern forms of collaboration with associates, which otherwise would not be possible.

Nothing new here! When we accompanied the implementation of Hummingbird at the Ministry of General Affairs in 2002-2003, we wrote - if not literally - the same.

Hummingbird is a system that we are currently asked to ‘phase out’. Turnover is fast!

RecognitionRecognition means: base the structure within SharePoint on your organization’s work practices. This will assist to organize your digital archive. Microsoft helps you on your way through delivery of templates for team sites, project sites and others.

SharePoint can only be successful if the organization recognizes the work practices SharePoint supports; and where at the same time digital archiving is automated (as much as possible ‘behind the scenes’).

How?Many organizations choose to organize their collaboration sites principle on process-orientation, that does not mean those organizations know only method of working. Of course not! Process orientation means ensuring that each method of working within the collaboration site is linked to the process architecture of the organization, and through that connection to an archive process.

Unlike working with a business oriented project, keep in mind that not all files are created within the organization in the same way. There is a big difference in file creation - especially in the perception of the end user - from carrying out a project, providing a grant, realization of administrative decisions or weekly meetings. There are different work practices that require different ways of digital support and documentation. For example, you may want to operate a project with more extensive files than files for a grant and you may want to work in a meeting with a continuous action list.

Keep the number of processes as limited as possible for the purpose of clarity and standardization. Generally, you come in (office) organizations into the following:

Process-oriented work

Project work

Working in teams

Meetings

Managerial decision

Share knowledge.

Implementation in PhasesWhen you implement SharePoint, you should have a plan. For years SharePoint was thrown into organizations as an extra to Microsoft Office by IT departments without thought or guidance. The results of this approach lead to increased fragmentation.

It is wise to divide the implementation of a collaboration site like SharePoint into clearly identifiable phases. For example, think of:

31

Intake PhaseThis phase is “putting your house in order”. You translate your principles for digital work to vision, policy frameworks, agreements and functional blocks. Call it conveniently ‘governance’. This governance is the basis for the standard SharePoint and thus also for functional design.

Activities may include, for example:

Establishing an implementation framework, including a Project Start Architecture (PSA), digital archiving and records management.

Develop and adopt a vision of digital collaboration and a digital workplace.

Identify work processes within the organization and the required functionality the collaboration site must deliver.

It is very important to endorse the translation at the right level. During the rollout, you will indeed receive many times questions like “Who decided this?” And “why is this set up this way?” Support from management prior to implementation is a must. This goes beyond words. Management needs to “Walk the talk”.

Preparatory and Establishment PhaseYou direct the collaboration site in the different modes of operation that occur within your organization. Every method requires a specific set of functionality.

Activities may include:

Drafting and determining the functional design. This can be in the form of a traditional description of functionality; it can also be in the form of ‘user stories’. At the executive level, describing what it wants a user to do in a particular role in SharePoint to support his/her work. For example: “As an archivist I want to see which files closed in a certain period.”

The user stories include the requests of end users and that of the organization as a whole. They are collected from “ordinary” users, but of course also from users with a “special role” in information management (such as the archivist).

Setting up the digital archive in the collaboration site. Digital archiving should describe functionality, which is determined in each case how documents, files, processes, selection criteria and the process of retention and deletion are associated. This is especially true for government agencies, but also the business community has an interest in good recordkeeping, including timely deletion.

Setting up a user organization. A balanced foundation of the user organization is desirable to support and to advise the project. For example, implementing functionality requests.

Pilot PhaseThe pilot phase you use to test whether what you’ve figured out works in practice. So a company’s project or process group goes to work in SharePoint and reports its findings. This would most likely lead the pilot to modification in technique, functionality, management and / or task change.

Roll Out PhaseThe focus of the roll out phase: as soon as possible everyone on SharePoint! A lot of attention is necessary to guide changes in attitude and behavior. By quickly moving all employees over to SharePoint and to close old storage systems users will find it less attractive to fall back into “old behavior”.

Incidentally, the manner in which you guide the change differs between organizations and organizational departments. An engineering department requires a different approach than the personnel department.

32

Out of scopeA wise project manager keeps the following activities as much as possible outside the scope of the site implementation:

Migration activities from old systems to the collaboration site.

Simultaneously implementing migration with the new inevitably can lead the project in the direction of “removal of old junk “rather than a transition to a new method of working. Old rubbish can be described by many organizations as thirty years of unhinged folder structures on the network, fifteen levels deep, and only focuses on the time convenience of the individual. Of course, migration is a major concern for the organization, but you should invest in specific projects focused on migration. Preferably, after the new site has been put to use.

Optimizing the development of the collaboration site.

You should limit your functionality scope. Implement a basic form of digital collaboration. For some users that moment is a step back but if you let ‘different functional releases’ be part of your project, your project will see huge delays through the endless discussions that arise. So: collect the functionality optimization requests that come up during the roll out in your file ‘Functionality Optimizations after Roll-out and preferably hand them over to operations. Operations can after the end of the project at a leisurely pace plan and implement these. This occurs through a structured and controlled process. The implementation has then become a part of the existing organization. The same story goes for the ‘specials’ you encounter in every organization; ways of working that are specific to a department or a process. Put these at the very end of your project or turn them over to operations.

Assuring effective use of SharePoint after the Roll Out phase.

A project organization has an obligation to maximize the use of the site. No more. The actual responsibility for sustainable use of SharePoint lies with the operation managers. Make sure that during the project regular transfer occurs of completed components to operations.

So far, Part 1 it gives an idea of how we work with the implementation of a collaboration site. In the Part 2 we discuss the aspects of the information architecture, the sense of quality, change assignment and the risks during the implementation of the collaboration site.

This is adapted from an article under the title “Implementing a collaboration site ... what do we need to think about?’ published in the book” A happy life and people working digitally 2015-2016.

The book (in Dutch) can be ordered through info@digital.nl.

Have you ever visited a DIWUG event (in Dutch)? Just register on the DIWUG site to receive the monthly newsletter.

eMagazine #18 - November 2016

33

Introducing the PnP Partner Pack v. 2.0by Paolo Pialorsi

The PnP Partner Pack is a reference solution provided by the SharePoint/Office 365 Developers Patterns and Practices (PnP) initiative (http://aka.ms/SharePointPnP). It is a sample Office 365

Application, which allows you to manage the most common and most frequently requested tasks, in the fields of handling sites and site collections in SharePoint Online.

It is a reference solution, because it uses and leverages most of the patterns and guidance promoted by PnP, and as such can be used not only as a ready to go solution, but also as a reference to take inspiration for any other business-level solution.

This article introduces the v. 2.0 release of the PnP Partner Pack, which PnP shipped in September 2016 and which is available at the following URL: http://aka.ms/OfficeDevPnPPartnerPack.

Main capabilitiesThe PnP Provisioning Engine is the foundation of the PnP Partner Pack, and the main objective of the solution is to support users and IT Pros to manage sites and sites collections creation, as well as to provide capabilities to save a site as a PnP provisioning template, without being an IT expert. Moreover there are some governance tools, which can be helpful for IT Pro managing a SharePoint Online tenant.

Here follow the most important features of the PnP Partner Pack v. 2.0:

Self-service site collection and sub site provisioning solution

Fully configurable based on business requirements

Save existing site as new template from the standard user interface

Template creation does not require xml or script knowledge - New templates can be generated from the existing sites

Sub site creation implementation with remote provisioning

Support for tenant wide or site collection templates

Responsive UI package for the Office 365 SharePoint sites

Uses JavaScript and custom CSS files to transform out of the box SharePoint sites as responsive

Can be applied to any SharePoint site and does not depend on the PnP Partner Pack

UI widget implementations with JavaScript embedding pattern to avoid custom master pages

Governance tools for administrators: apply SharePoint farm-wide branding, refresh site templates, bulk creation of site collections

Reference governance remote timer jobs (Azure WebJobs) to perform typical enterprise governance operations to existing site collections and sites

Configurable branding and text elements for easy branding element changes

The main capability of the PnP Partner Pack is the self-service creation of sites and site collections, which allows end users to create sites without the need to interact with the IT department. However, the sites created will be based on a set of PnP provisioning templates defined by the IT of the company. Moreover, as a fresh new feature introduced with the v. 2.0 of the solution, users can now search for provisioning templates available in a free and public gallery called PnP Templates Gallery.

34

The PnP Templates Gallery is a web site (https://templates-gallery.sharepointpnp.com) that allows the entire community of Office 365 to share open source templates for sites and site collections. In the following Figure 1 you can see the user interface of the PnP Partner Pack while creating a new site collection.

Figure 1: The UI of the PnP Partner Pack while creating a new site collection

As you can see it is possible to select the templates provider and to make a free text search for templates. Out of the box, the templates providers available for creating site collections are a tenant-level provider, and the PnP Templates Gallery. However, you can configure whatever provider you like, because under the cover there is an open model. Of course, and if you like, you can also disable any of the out of the box providers. For example, if you don’t want your users to search on the public gallery, you can easily turn off that capability.

When the users want to create a sub site in an already existing site collection, they still do have the capability to search templates at tenant-level and in the gallery. Moreover, there is also the capability to store the templates at the site collection level, so that site collection administrators can create their own local library of templates, if needed.

Another killer feature of the PnP Partner Pack is the “Save Site as Provisioning Template” command, which is embedded in the Site Settings of any sites, on which you enable the PnP Partner Pack extensibility model. In Figure 2, you can see the user interface of this functionality.

Figure 2: The UI of the PnP Partner Pack while saving a site as a provisioning template

35

We would be very gladto receive your article

on SharePoint forour next eMagazine.

The “Save Site as Provisioning Template” is a fundamental capability, because it allows end users – again without direct interaction with the IT department – to save any live and already existing site as a template, which can afterwards be used to create other sites with the same artifacts.

All these functionalities are embedded into the native user interface of SharePoint Online, and support the new modern UI of SharePoint. Thus, the users will have a fluent and smooth interaction with the solution.

Moreover, all the long running operations like sites creation, templates generation, etc. are executed by an asynchronous job engine, which acts in background. This approach allows to have a fast interaction for the end users, and a monitorable execution process for actions from an IT perspective.

Another interesting capability included in the v. 2.0 of the solution, is the “Refresh Sites” command, which allows to keep in sync the sites created with the PnP Partner Pack with their templates. Basically, when you create a site with the PnP Partner Pack and later you update the template that you used for creating the site, you will able to update that site, too, keeping all your assets aligned. This is really helpful whenever you add lists, libraries, content types, or artifacts in general to a template and you want to keep in sync all the already created sites.

There are many other functionalities, which cannot be covered here for the sake of brevity, but you are invited to install the PnP Partner Pack on your tenant, and to play with it.

Requirements and SetupTo use the PnP Partner Pack, you will need to have an active subscription in an Office 365 tenant and a valid Azure AD subscription to host the application.

In fact, the application is based on an ASP.NET MVC web site that is planned to be hosted on an Azure App Service. Of course, you can host it wherever you like, but the official setup guide of the product (https://github.com/OfficeDev/PnP-Partner-Pack/blob/master/Documentation/Manual-Setup-Guide.md) targets Microsoft Azure.

To setup the solution you have to satisfy the following fundamental steps:

Register the application in Azure AD, within the tenant that sits under the cover of your Office 365 subscription

Create a self-signed certificate to configure App-Only access to the SharePoint Online environment

Create an infrastructural site collection in the target SharePoint Online, which will be used to store the tenant-wide templates

Create an Azure Blob Storage account

Create an Azure App Service to publish the web application

Add some Azure WebJobs to run the asynchronous tasks and the governance jobs

Luckily, in the PnP Partner Pack step-by-step setup guidance mentioned above, you will find all the steps explained, as well as a bunch of PowerShell scripts that will do all the actions for you. It is also under development a desktop application for Windows, which will provide a setup wizard to make much more easy to setup the solution.

In the GitHub repository that hosts the PnP Partner Pack, there is also a reference to a video guide, that you can follow to setup the product.

36

Architectural PillarsFrom an architectural perspective, the PnP Partner Pack leverages some interesting patterns that are illustrated in the following paragraphs.

First of all, the solution is defined as an Azure AD application (or Office 365 Application), which makes it possible to access the entire ecosystem of services provided by Office 365, including the powerful Microsoft Graph API. From an Azure AD perspective, the application is registered to run both with a delegated access, on behalf of the current user, as well as with an App-Only token. The App-Only token requires to register the public key of an X.509 certificate in Azure AD, so that the application will be able to use its private key to sign the OAuth tokens sent to Azure AD.

From a users’ authorization perspective, the solution uses the Microsoft Graph API to search for users and their membership at the tenant level. For example, all the settings and the governance features are available only to tenant users configured as Tenant Admin or SharePoint Admin.

Moreover, the asynchronous processing of tasks uses an Azure Blob Storage Queue that activates an Azure WebJob, whenever there is a new message in the queue. In Figure 3 you can see the architecture of this asynchronous model.

Figure 3: The architecture of the asynchronous model for the Azure WebJob

Another interesting feature of the PnP Partner Pack is that the full UI of the solution is built using the new Office UI Fabric JS components (http://dev.office.com/fabric). The UI is responsive using the Office UI Fabric grid, and all the MVC editor controls used in the UI have been built using the HTML/CSS/JS widgets provided by the Office UI Fabric project.

As such, the solution provides a user interface and a user experience really close to the native one of Office 365, giving to the end users the idea of a fully integrated solution. Just for the sake of making an example, whenever the users have to select a user principal, like for example a site collection administrator for a site under creation, they will interact with the Office UI Fabric PeoplePicker control, which has been incorporated in an MVC editor control, and which uses the Microsoft Graph API to search for users. In Figure 4 you can see the UI of the PeoplePicker control.

37

Figure 4: The UI of the PeoplePicker control while creating a new site collection

Last but not least, the architecture of the project is completely open. For example, the solution targets SharePoint Online only, but you can implement a bunch of .NET interfaces available in the project, register you custom types, and you can create a customized version of the project that targets SharePoint on-premises, or eventually a hybrid topology. On the GitHub repository of the solution, there is an architectural guide (https://github.com/OfficeDev/PnP-Partner-Pack/blob/master/Documentation/Architecture-and-Implementation.md) that explains you what are the extensibility points and how to create a customized setup of the solution.

There are many other capabilities, which are implemented in the PnP Partner Pack, and that leverage other dedicated Azure WebJobs. More in general the architecture of the solution shows off what you can leverage, and how you can leverage the services provided by Microsoft Azure and the power of the Office 365 ecosystem.

ConclusionThe goal of this article was to briefly introduce you to the PnP Partner Pack v. 2.0, which is an open source solution that you can use as such, or that you can customize based on your needs. The main objective of the solution is to give inspiration and to show how to leverage the patterns and the tools promoted by PnP. Install it, play with it, contribute and give feedbacks to the PnP Core Team using the GitHub issues and pull requests!

eMagazine #18 - November 2016

38

Microservices: The good, the bad and the uglyby Sander Hoogendoorn

Back in 1988, when I was first employed by a company for writing software, the world was fairly simple. The development environment we had was character-based, the database was integrated and traversed with cursors, and we built a whole new administrative system covering everything

but the kitchen sink. It took us five years to complete the project, basically because the client kept changing his mind every now and then, and because every change in the code could break code

elsewhere in the system. Unit testing hadn’t been invented yet, and testing was done by the end users. In production!

So far for monoliths. Then in 1994 I joined a company that build desktop applications – remember the world wide web was only a couple of years old, and web applications hadn’t been invented yet. We used a great tool called PowerBuilder and now we had two components to worry about; the application on the desktop and the database on the server. The applications we build usually served departments or sometimes even a whole company. Not highly complicated, but not highly scalable either. Ah well, we had fun as long as the client-server paradigm lasted.

Component based developmentThe world got more complex in the mid-nineties. Companies wanted web applications, basically running on intranets to get rid of desktop deployments. And applications needed to serve multiple departments, and sometimes even break company borders. A new paradigm set in: component based development, also known as CBD. The paradigm promised us re-use, scalability, flexibility and a way to harvest existing code (usually written in COBOL). We started breaking up our systems into big functional

Figure 1: Scary stuff, an object request broker architecture

chunks, and tried hard to have those components communicate to each other. Java was invented and everybody now wanted to write code in Java (apparently some people still to this nowadays). Components where running in impossible technologies

39

such as application servers and CORBA (to impress your co-workers, look that one up on Wikipedia). The good old days of object request brokers!

At that time I was working for a large international bank trying to set up an methodology for doing component based development. Even with a heavily armed team of Anderson consultants it took us three years to write the damn thing. In the end both the paradigm and the technology where too complex to write decent and well-performing software. It just never took off.

Service oriented architectureAt that point in time, the early years of this century, I thought we got rid of distributed software development, and happily started building web applications. I guess everyone bravely ignored Martin Fowler’s first law of distributed objects: do not distribute your objects. Gradually we moved into the next paradigm of distributed computing, re-packaging the promises of component based development into a renewed set of technologies. We now started doing business process modeling (BPM), and implemented these processes in enterprise services buses (ESB’s), with components delivering services.

We were in the age of service oriented architecture, better known as SOA.

Coming from CBD, SOA seemed easier. As long as your components – the producers – were hooked into the enterprise service bus, we figured out how to build-up scalable and flexible systems. We now had much smaller components that we could actually extract from our existing systems (now not only written in COBOL, but also in PowerBuilder, .NET and Java). The mandatory design patterns books where published and the world was ready to go. This time we would crack it!

Figure 2: An enterprise service bus

I found myself working for an international transport company, and we happily build around SAP middleware – delivering both ESB and BPM tooling. We now not only needed regular Java and .NET developers, but we employed middleware developers and SAP consultants as well. And although agile was introduced to speed up development (I know, this is not the right argument), projects still suffered from

40

sluggishness and moreover, when we got pieces of the puzzle in place, we started to realize that integration testing and deployment of new releases got more complicated by the day.

At last: microservices!I do hope you forgive me this long and winding introduction to the actual topic: microservices. You might think: why do we need yet another article on microservices? Isn’t there enough literature already on the topic. Well yes there is. But if you look closely to the flood of articles that you find on the internet, most of them only describe the benefits and possibilities of microservices (sing hallelujah), some of them take a look at the few famous path finding examples (Netflix, Amazon, and Netflix, and Amazon, and Netflix…). Only a few articles actually dig a bit deeper, and they usually consist of summing up the technologies people seem to be using when implementing microservices. It’s still early in the game.

That’s were a little historical perspective won’t hurt. I find it interesting to witness that the benefits and possibilities of the predecessors of microservices are still with us. Microservices seem to promise scalable and flexible systems, based on small components that can easily be deployed independently, and thus promote use of the best choice in technologies per component. Basically the same promises we fell for with CBD and SOA in the past. Nothing new here, but that doesn’t mean that microservices aren’t worthwhile investigating.

Is it different this time around?So why is it different this time? What will make microservices the winning paradigm, where its predecessors clearly were not? What makes it tick? As a developer, there’s no doubt that I am rather enthusiastic about microservices, but I was enthusiastic as well (more or less) when people came up with CBD and SOA.

I do suppose there are differences. For the first time we seem to have the technology in place to build these type of architectures. All the fancy and complex middleware is gone, and we rely solely on very basic and long-time existing web protocols and technologies. Just compare REST to CORBA. Also we seem to understand deployment much better, due to the fact that we’ve learned how to do continuous integration, unit testing, and even continuous delivery. These differences suggest that we can get it to work this time around.

Still, from my historical viewpoint some skepticism is unavoidable. Ten years ago, we also really believed that service oriented architecture would be technologically possible, it would solve all our issues, and we would be able to build stuff faster, reusable and more reliable.

So, to be honest, the fact that we believe that the technology is ready, is not much of an argument. Meanwhile the world also got more complex.

Over the last year I’ve been involved with a company that is moving away from their mainframe (too expensive) and a number of older Java monoliths (too big, and hard to maintain). Also time-to-market plays an important role. IT need to support introducing a new product in months, if not in weeks. So we decided to be hip and go microservices. Here’s my recap of the good, the bad and the ugly of microservice architectures, looking back on our first year on the road.

The goodLet’s start with the good parts. We build small components, each offering about two to six services. Good examples of such small components are a PDF component, that does nothing more than generate PDF’s from a template with some data, or a Customer component that allows users to search for existing customers. These components offer the right size. The right size of code, the right size of understandability, the right size to document, to test and to deploy.

41

Figure 3: The Good

Our team tend to evolve towards small teams designing, implementing and supporting individual components. We didn’t enforce ownership, but over time small teams are picking up the work on a specific component and start feeling responsible for it.

When we outlined the basic architecture for our microservices landscape we set a number of guidelines. We are not only building small components, but are also building small single-purpose web applications. Applications can talk to other applications, and can talk to components. Components handle their own persistence and storage, and can also talk to other components. Applications do not talk directly to storage. Components do not talk to each others storage. For us these guidelines work.

Microservices lives up to some of its promises. You can pick the right technology and persistence mechanisms for each of you components. Some components persist to relational databases (DB2 or SQLServer), others persist to document databases (MongoDB in our case). The hipster term here of course is polyglot persistence.

We also stated that every application and component has it’s own domain model. We employ the principles and patterns of domain driven design straightforward. We have domain objects, value objects, aggregates, repositories and factories. Because our components are small, the domain models are fairly uncomplicated, and thus maintainable.

Although we have had quite a journey towards testing our components and services, from Fitnesse to hand-written tests, we are now moving towards testers specifying tests in SoapUI, which we are running both as separate tests and during builds. We had to learn to understand REST, but we’ve got this one figured out for now. And testers love it.

When we started our microservices journey, the team I’m was in a highly reactive mode – tell us what to program and how to program it, and we’ll do our job. I probably won’t have explain what that does with the motivation of the team members. However, with microservices there is no such thing as a ready-to-use cookbook or a predefined architecture. It simply doesn’t exist. That means that we continuous find ourselves solving new pieces of the microservices puzzle, from discovering how to design microservices (we use smart use case), to how implement REST interfaces, how to work with a whole new set of frameworks, and a brand new way of deploying components. It’s this puzzle that make working on this architecture interesting. We allow ourselves to learn everyday.

42

The badBut as always, there’s a downside. When you start going down the microservices road – for whatever you may find it beneficial – you need to realize that this is all brand new. Just think of it: if you’re not working for Netflix or Amazon, who do you know who actually already does this already? Who has actively deployed services into production, with a full load? Who can you ask?

Figure 4: The Bad

You need to be aware that you really need to dig in and get your hands dirty. You will have to do a lot of research yourself. There are no standards yet. You will realize that any choices you now make in techniques, protocols, frameworks, and tools is probably temporarily. When you are learning on a daily basis, newer and better options become available or necessary, and your choices will alter. So if you’re looking for a ready-made IKEA construction kit for implementing microservices the right way, you might want to stay away from microservices for the next five to seven years. Just wait for the big vendors, they will jump in soon enough, as there’s money to make.

From a design perspective you will have to start to think differently. Designing small components is not as easy as it appears. What makes a good size component? Yes, it has a single business purpose, undoubtedly, but how do you define the boundaries of your component? When do you decide to split a working, operational component into two or more separate components? We’ve come across a number of these challenges over the past year. Although we have split up existing components, there are no hard rules on when to actually do this. We decided mainly based on gut feeling, usually when we didn’t understand a components structure anymore, or when we realized it was implementing multiple business functions. It gets even harder when you are chipping off components from large systems. There’s usually a lot of wiring to cut, and in the meantime you will need to guarantee that the system doesn’t break. Also, you will need a fair amount of domain knowledge to componentize your existing systems.

Basically we found that components are not as stable as we first thought. Occasionally we merge components, but it appeared far more common to break components into smaller ones to provide reuse and shorter time-to-market. As an example, we pulled out a Q&A component from our Product component that now supplies questionnaires for other purposes than just about products. And more recently we created a new component that only deals with validating and storing filled-in questionnaires.

43

There’s lots of technical questions you will need to answer too. What’s the architecture of a component? Is an application a component as well? Or, a less visible one, if you are thinking of using REST as your communication protocol – and you probably are – is: how does REST actually work? What does it mean that a service interface is RESTful? What return codes do you use when? How do we implement error handling in consumers if something isn’t handled as intended by one of our services?

REST is not as easy as it appears. You will need to invest a lot of time and effort to find out your preferred way of dealing with your service interfaces. We figured out that to make sure that services are more or less called in a uniform way, we’d better create a small framework that does the requests, and also that deals with responses and errors. This way communication is handled similar with every request, and if we need to change the underlying protocol or framework (JAX-RS in our case), we only need to change it in one location.

That automatically brings me to the next issue. Yes, microservices deliver on the promise that you can find the best technology for every component. We recognize that in our projects. Some of our components are using Hibernate to persist, some are using a MongoDB connector, some rely on additional frameworks, such as Dozer for mapping stuff, or use some PDF-generating framework. But with additional frameworks comes the need for additional knowledge. At this company, we will easily grow to over a hundred small components and maybe even more. If even only a quarter of these use some specific framework, we will end up with twenty-five to thirty different frameworks (did I mention we do Java?). We will need to know about all of these. And even worse, all of these frameworks (unless they’re dead) version too.

Then there’s the need to standardize some of the code you are writing. Freedom of technology is all good, but if every component is literally implemented differently, you will end up with an almost unmaintainable code base, especially since it is likely that no-one oversees all the code that is being written. I strongly suggest to make sure there’s coherence over your components on aspects of your code base that you could and probably should unify. Think of your UI components (grids, buttons, pop-ups, error boxes), validation (of domain models), talking to databases or how responses from services are formulated. Also, although I strongly oppose having a shared domain model (please don’t go that route), there’s elements in your domain models you might need to share. We share a number of enumerations and value objects for instance, such as CustomerId or IBAN.

If you’re a bit like us this generic code ends up in a set of libraries – a framework if you will – that is reused by your components. We have learned that with every new release of this home-grown framework we end up refactoring some of the code of our components. I rewrote the interface of our validation framework last week, which was necessary to get rid of some state it kept – components need to be stateless for clear reasons of scalability – and I’m a bit reluctant to merge it back into the trunk when I get back to work after the weekend. Most of our components use it, and their code might not compile. I guess what I’m saying here is that it’s good to have a home-grown framework. With some discipline it will help keep your code somewhat cleaner and a tad more uniform, but you will have to reason about committing to it and releasing new versions of it

The uglySo what about the really nasty parts of microservices? Let’s start with deployment pipelines. One of the promises of microservices is that they can and should be individually deployed and released. If you are currently used to having a single build and deployment pipeline for the one system you are building or extending, you might be in for a treat. With microservices you are creating individual pipelines for individual components.

44

Figure 5: The Ugly

Releasing the first version of a component is not that hard. We started with a simple Jenkins pipeline, but are currently investigating TeamCity. We have four different environments. One for development, one for testing, one for acceptance and of course the production environment. Now we are slowly getting in the process of releasing our components, most of them with their own database, we start to realize that we cannot do this without the support and collaboration of the operations team. Our expectations are that we will slowly evolve into a continuous delivery mode, with operations incorporated in the team. Right now we already have enough trouble getting operations on-board. They are now used to quarterly releases of the whole landscape, and certainly have no wish for individually deploying components.

Another that worries me is versioning. If it is already hard enough to version a few collaborating applications, what about a hundred applications and components each relying on a bunch of other ones for delivering the required services? Of course, your services are all independent, as the microservices paradigm promises. But it’s only together that your services become the system. How do you avoid to turn your pretty microservices architecture into versioning hell – which seems a direct descendant of DLL hell?

To be honest, at this point in time, I couldn’t advice you well on this topic, but a fair warning is probably wise. We started out with a simple three-digit numbering scheme (such as 1.2.5). The rightmost number changes when minor bugs have been solved. The number in the middle raises when minor additional functionality has been added to a component, and the leftmost number changes when we deploy a new version of a component with breaking changes to its interface. Not that we strongly promote regularly changing interfaces, but it does happen.

Next to that, we test our services during the build, both using coded tests and tests we assemble in SoapUI. And we document the requirements in smart use cases and domain models of our applications and components using UML. I’m quite sure in the future we need to take more precautions to keep our landscape sane, such as adding Swagger to document our coded services, but it’s still to early in the game to tell.

The hockey stick modelEarly in the game? Yes, we’ve only been on the road to microservices for about a year. And I still haven’t figured out whether we are on a stairway to heaven or on a highway to hell. I do suppose, as with it’s historical predecessors, we will end up somewhere in the middle, although I really do believe that we do have the technology to get this paradigm working – and with we I don’t just mean Netflix, Amazon or some hipster mobile company, but the regular mid-size companies you and I work for.

45

But will it be worthwhile? Shorten time-to-market? Deliver all the goodies the paradigm promises us? To be honest, I don’t know yet – despite or maybe even because of all the hype surrounding microservices. What I did notice is that, given the complexity of everything surrounding microservices, it takes quite a while before you get your first services up and running, and we are only just passing this point. Some weeks ago, I was having a beer with Sam Newman, author of Building Microservices. Sam confirmed my observations from his own examples and referred to this pattern as the hockey stick model.

Figure 6: The hockey stick model

There’s a lot to take care of before you are ready to release you first service. Think of infrastructure, sorting out how to do REST properly, setting up you deployment pipelines, and foremost change the way you think about developing software. But as soon as the first service is up and running, more and more follow faster and faster.

Be patientSo, if there’s one thing I’ve learned over the past year, it is to be patient – a word which definitively did not appear in my dictionary yet. Don’t try to enforce all kinds of (company) standards if they just don’t exist yet. Figure it out on the fly. Allow yourself to learn. Take it step by step. Simply try to do stuff a little bit better than the day before. And as always, have fun!

eMagazine #18 - November 2016

46

Office 365 collaboration tools: what to use for what today?

Frédérique Harmsze works as a consultant at Macaw. She has been doing SharePoint projects since 2004, both in small organizations and in international teams for multinationals. In her work, she focuses on the client and the end-user, rather than the technical aspects. She plays a role in the implementation, from the functional angle, of SharePoint and Office 365 and is often involved in user adoption. She blogs about it on her website http://frederique.harmsze.nlCompany: Macaw - The Netherlands

Visual Studio Team Services Release Management for O365 product development

Clemens Reijnen in short:Sogeti Labs Fellow | Management Consultant | Speaker | Blogger | Software Developer | Agile ALM Coach | Cloud Solution Architect | Free Heel Skier | RunnerBlog: www.clemensreijnen.nlTwitter: @ClemensReijnenEmail: clemens.reijnen@sogeti.com

Information management Office 365

Albert Hoitingh is business consultant working for the Microsoft business unit at Sogeti Netherlands. He’s been working with information worker solutions from the late 1990’s, starting with IBM Lotus Notes. He has extensive experience with Microsoft SharePoint (2001-2013, Online) as-well-as other Microsoft platforms. Albert enjoys inspiring organizations as a SharePoint evangelist and works closely with business users to successfully implement SharePoint solutions. He’s presented at SharePoint Connections in Seattle and in Amsterdam as well as the Sogeti SharePoint events.When he’s got the time, he blogs at alberthoitingh.wordpress.com.Company: Sogeti - The Netherlands

Implementing SharePoint as a collaboration site

Ronald Groeneweg is managing partner of Digitalemail: ronald.groeneweg@digital.nlblog: http://www.digital.nl/nieuwstwitter: @digitaalwerken

Robert Wetting is managing partner of Informatiemanagement Noord. Hij heeft functies bij de (semi) overheid, bedrijfsleven en verzekeraars vervuld in vele verschillende rollen: adviseur, interim leidinggevende, trainer, verandermanager, functioneel consultant en inhoudelijk deskundige.email: robert.wetting@imnoord.nlblog: http://imnoord.nl/nieuws/twitter: @imnoord

About the Authors

47

Microservices. The good, the bad and the ugly

Sander is an independent consultant, software architect and programmer, experienced in (beyond) agile, Scrum, Kanban, continuous delivery, software estimation, agile requirements, UML, software architecture, microservices, design patterns, and both Java and .NET development.Sander authored best-selling books “This Is Agile” and “Pragmatisch Modelleren Met UML” and published well over 250 articles in international magazines. He is an inspiring (keynote) speaker at international conferences on topics. He also presented over 300 international (in-house) training courses and lectured at many universities.email: getintouch@ditisagile.nlblog: http://sanderhoogendoorn.com/blog/index.php/test/t: @aahoogendoorn

Introducing the PnP Partner Pack v. 2.0

Paolo is a consultant, trainer, book author and speaker at the best international conferences about Microsoft technologies. He works in a company of his own, called PiaSys.com, and he is focused on Microsoft Office 365 and Microsoft SharePoint.He writes articles for IT magazines, authored several books for Microsoft Press and posts regularly on his technical blog (http://www.sharepoint-reference.com/blog) and Twitter (@PaoloPia).Paolo passed about 50 Microsoft certification exams including the Microsoft Certified Solutions Master - Charter SharePoint. In 2015 was awarded as MVP on Microsoft Office 365.Since 2015 he is member of the Core Team of the Office 365 Developer Pattern and Practices project.

Samen de kar trekkenSinds het eerste nummer in januari 2010 verscheen is het SharePoint eMagazine van DIWUG een stabiele factor gebleken in de IT-wereld van IT-pro’s, developers en end (power) users. Elke editie wordt telkens weer meer dan 7.000 keer gedownload (Acrobat versie). En ondanks enkele technische beperkingen wordt zelfs de e-reader versie van elke uitgave zo’n 2.500-3.000 keer van DIWUG.nl opgehaald.

Met ingang van no. 4 (januari 2011) wordt elke editie in een beperkte oplage gedrukt. Meestal moest de redactie al een week na uitgifte vele aanvragers teleurstellen. Hoewel een boegbeeld voor de papierloze samenleving (!) gingen de gedrukte eMagazines in razend tempo over de toonbank. Pakketten van een aantal exemplaren werden over de hele wereld verzonden. IT-afdelingen van multinationals en instellingen gebruiken het eMagazine als onderdeel van hun “Levenlang Leren”-programma.

Bijna 7 jaar geleden opgezet door drie enthousiaste SharePoint MVP-ers wordt de kar nog steeds door vrijwilligers getrokken. Zonder subsidie. Geheel op basis van fondsenwerving binnen de IT-branche van SharePoint. Hetzij in de vorm van advertenties, vergoeding van de drukkosten of redactionele artikelen onder de bedrijfsnaam.

Als bijkomende tegenprestatie zorgt DIWUG ervoor dat alle advertenties en bedrijfsgegevens in de Acrobat-versie via hyper- of cross- links naar de website van de sponsor leiden.

DIWUG maakt graag van deze gelegenheid gebruik om alle sponsoren en schrijvers van de afgelopen 7 jaar te bedanken voor hun bijdrage tot het succes van dit eMagazine.

De redactie hoopt de komende jaren weer een beroep te mogen doen op de professionele kennis van MVP-ers en het zakelijk inzicht van IT-bedrijven waarbinnen SharePoint een toonaangevende rol vervult.

Bij voorbaat dank voor uw toekomstige bijdrage!

De redactie.