Using DFS and GPO in FIM High Availability Scenarios

Brad TurnerSolutions Architect, Ensynch

http://www.identitychaos.com

Ensynch Introduction• Infrastructure and Applications IT Consulting Firm (10+ Years in Business)

– Pure Play Services Organization --- we don’t sell software, hardware – National Award Winning Microsoft Gold Certified Partner– Quest Software Partner – #1 National Services Partner for Year to Year Growth

• Advanced Competencies Across the Microsoft IT Stack– Core Infrastructure, Business Productivity, Application Platform / Business Optimization– Credentialed Microsoft Experts: MVPs (ILM), VTSs (BizTalk), Solution Specialists (VSSPs)

• Additional Unique Capabilities– Datacenter Expertise Enhances our Cloud Readiness– Mature Project Management Organization uniquely set up to handle all your project needs– Resourcing and Technical Staffing Services Organization

• Geographies – Corporate HQ in Tempe, AZ. Major presence in NY, NJ and So-Cal: OC, LA, San Diego.

Additional key customers across North America as well as Europe

• 185+ co-workers strong, and currently hiring

Ensynch’s Identity & Secure Access Mgmt PracticeOur ISAM Practice Has a Reputation for Excellence• World Renowned for our Enterprise Identity + SSO Business Accelerator

Solutions• Commonly complete successful projects for organizations ranging from Fortune

500 to Upper Mid-Market to Education • Credentialed Identity Management Thought Leaders, 2 Microsoft Identity Mgmt

MVPs• Quest and Microsoft Identity Mgmt Solution Expertise• Already Implemented FIM2010 for multiple organizations prior to RTM launch • Wrote Microsoft’s official FIM2010 Technical Overview and Custom Workflow

Whitepapers

Brad Turner, Solutions Architect• 15+ years in consulting, 10 years in IDA• Pretending I have time for family and squeezing in time for Xbox (idcha0s)Ensynch’s ISAM Business Accelerator Solutions (Offering Categories)

Directory Services

Identity ManagementAutomation

Overview

Problem statement

• We need to replicate configuration data not in the Metaverse

High Availability Options• Sync• SQL• Portal• Failover scripting with Mirrored partner

Implementing Best Practices via GPO• Account Lockdown• GPO Preferences

Data Replication using DFS• DFS Namespaces• DFS Replication Groups

Problem statement

What doesn’t exist in SQL that I need to keep?

• MAData items• XMA or Extension configuration files• CS or DSML reports• Dependent code libraries• Scripts• Scheduled Tasks!

What HA options are available?

• SQL Databases• Sync Engine (FIM/ILM)• FIM WS/Portal

So, about those HA options, we need flexibility…

SQL High Availability

Failover Clustering

• SQL Failover by Instance• All SQL Databases support clustering• Support for GenericScript resource allows for flexibility

Database Replication

• Mirroring – faster recovery but not all apps support it• Log Shipping – automated Tlog backup and copy

What about the Sync Service?

Sync Engine (FIM/ILM/MIIS)

Tightly coupled with the server

• Does not support automatic failover via mirroring• Mirroring and Log Shipping are transparent to the app, but recovery of the app is manual

Warm Standby Server

• The “official” solution, requires restoration of the database and run miisactivate to update the internal pointers• Locating a mirror or log shipped copy of your db on the warm standby server is the same as restoring the db – still have to run miisactivate

Configuration Data

• Still need to replicate MAData and other config items on the File SystemWhat options are available for FIM Service?

FIM Web Service and Portal

Web Tier Scale Out Model

• Multiple instances of the FIM WS are supported• When combined with the Portal application it scales out nicely• Load balancing the web tier provides fault tolerance and capacity

WSS Web Farm Mode

• Farm Mode handles replication of portal content between web servers in the farm• FIM Web Portal installed only once• FIM Web Service installed on each node in the farm• Kerberos is essential!

So, lets look at an example…

FIM Sync – Warm Standby

• Standard Sync Warm Standby scenario with SQL Cluster• Failover of FIM Sync is manual

What if I can’t cluster?

FIM Sync – Non-clustered

• When clustering isn’t an option, mirroring provides HA

• Sync Service is still manually failed over

• DFS Replication used to keep FIM Folders (MAData) in sync

• Group Policy Preferences used to replicate Tasks

• Protects against local drive corruption

Can I do both Mirroring and Log Shipping?

FIM Sync – Mirroring and Log Shipping

• With FIM DB’s use High Performance, asynchronous mode for mirroring• For dependent databases you may choose High Safety, synchronous• It is possible to do both Log Shipping and Mirroring at the same time

Now for the big picture…

FIM HA – Multi-instance

Any guidance for HA Sync Services?

Clustered – Sync Service on local nodes

• Install ILM/FIM on the shared drive so your data fails over with the database• Move Run Profile scripts from Scheduled Tasks to SQL Agent• Still need to account for GAC updates via script• Miisactivate can be scripted as part of the cluster GenericScript resource

but it’s tricky• When referring to the server the Sync Service is installed on, make sure you

use the virtual server name and not the SQL host

Non-Clustered – Collocated Sync and DB

• Install ILM/FIM on both primary and standby servers• Use Database Mirroring or Log Shipping to copy the db to the second node

(Async Mode!) – or rely on manual restores to the standby• Use DFS to replicate the MAData and Scripts content to the standby• Use GPO Preferences to publish Scheduled Tasks to both sides• Use a manual failover script – lots of stuff to forget!

Test Graceful failovers during maintenance cycles!!!!

HA Sync Suggestions

What about Virtualization?

HA Sync Suggestions - Virtual

Virtual HA

• Microsoft supports the big vendors• Protects against hardware layer failures by shifting the image to a new host• Like clustering – does not protect against data corruption• No protection against OS/APP layer corruptions• Simple low cost/low complexity solution

Lets look at a failover scenario…

Moving to the standby server

miisactivate

• Still miisactivate in FIM 2010, same process• "d:\program files\microsoft identity integration server\bin\miisactivate.exe" G:\Backups\Keyring\

ilmkeyring.bin %USERDNSDOMAIN%\svc.ilmsync *

Don’t forget PCNS

• admod -b "CN=fimprimary,CN=Password Change Notification Service,CN=System,DC=dom,DC=com" "MS-MIIS-PCNS-TargetDisabled::TRUE -exterr

• admod -b "CN=ilmstandby,CN=Password Change Notification Service,CN=System,DC=dom,DC=com" "MS-MIIS-PCNS-TargetDisabled::FALSE -exterr

Verifying PCNSC:\Program Files\Microsoft Password Change Notification>pcnscfg list

The service configuration is not set. Defaults will be used by the service.

 <SNIP>

 

Targets

  Target Name...........: fimprimary

  Target GUID...........: 88B2A357-38A0-4AA2-8AA3-C41AF6AF3314

  Server FQDN or Address: fimprimary.dom.com

  Service Principal Name: CIMSPCNSCLNT/fimprimary.dom.com

  Authentication Service: Kerberos

  Inclusion Group Name..: DOM\Domain Users

  Exclusion Group Name..: DOM\PCNSDoNotNotify

  Keep Alive Interval...: 0 seconds

  User Name Format......: 3

  Queue Warning Level...: 0

  Queue Warning Interval: 30 minutes

  Disabled..............: True

 

  Target Name...........: ilmstandby

  Target GUID...........: 5630B0B1-6799-4A64-9157-2899465B97B0

  Server FQDN or Address: ilmstandy.dom.com

  Service Principal Name: CIMSPCNSCLNT/ilmstandby.dom.com

  Authentication Service: Kerberos

  Inclusion Group Name..: DOM\Domain Users

  Exclusion Group Name..: DOM\PCNSDoNotNotify

  Keep Alive Interval...: 0 seconds

  User Name Format......: 3

  Queue Warning Level...: 0

  Queue Warning Interval: 30 minutes

  Disabled..............: False

 

Total targets: 2

And now for some code overview…

Scripting Failover - Overview

This is not cluster failover• Disable Primary PCNS target• Disable scheduled tasks on Primary (assumes graceful transfer)• Force any running tasks on Primary to stop• Stop the Sync Service – kill if hung

Shutdown Primary• Update the GAC

Update Standby• Failover the mirror or restore the database

Failover or Restore SQL on Standby• Enable Standby PCNS target• Run miisactivate• Change the Sync Service to auto start• Start the Sync Service• Enable dormant scheduled tasks

Activate Standby

Scripting Manual Failover – Shutdown Primary

@echo off

echo B E G I N  P C N S  C U T O V E R

echo ----------------------------------------------------------

echo Failing over the PCNS targets, this process MUST complete prior to running MIISActivate

admod -b "CN=ilmstandby,CN=Password Change Notification Service,CN=System,DC=dom,DC=com" "MS-MIIS-PCNS-TargetDisabled::FALSE" -exterr

 

echo B E G I N  G R A C E F U L  S H U T D O W N  O F  P R I M A R Y

echo ----------------------------------------------------------

echo Disabling scheduled tasks on the Primary

schtasks /Change /S fimprimary /TN “FIM-Delta-Loop" /Disable

schtasks /Change /S fimprimary /TN "Daily-Maint" /Disable

schtasks /Change /S fimprimary /TN "ClearFIMRuns" /Disable

 

echo Forcing scheduled tasks on the Primary to stop

schtasks /End /S fimprimary /TN “FIM-Delta-Loop"

schtasks /End /S fimprimary /TN "Daily-Maint"

schtasks /End /S fimprimary /TN "ClearFIMRuns"

echo Graceful shutdown - stop FIMSync on Primary

sc \\fimprimary config FIMSynchronizationService start= demand

sc \\fimprimary stop FIMSynchronizationService

echo Waiting for 10 seconds to make sure service has stopped

sleep 10

echo Killing the service if it's stuck

taskkill /F /S fimprimary /IM miiserver.exe /T

Scripting Failover – Update Standby

echo B E G I N  S T A N D B Y  U P D A T E  P R O C E S S

echo ----------------------------------------------------------

echo Registering the Middleware libraries in the GAC

dir D:\ILMTasks\ILMConfig\MiddlewareGAC\*.dll /b >D:\ILMTasks\ILMConfig\MiddlewareGAC\assemblyList.txt

"C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\gacutil.exe" /il D:\ILMTasks\ILMConfig\MiddlewareGAC\assemblyList.txt /f

Keep copies of your libraries in a directory for handy GAC registration!

Critical if you have local FIM Service instances and custom WF!

Scripting Failover – SQL Failover

echo B E G I N  S Q L  F A I L O V E R

echo ----------------------------------------------------------

echo.

sqlcmd -S fimprimary -E -d master -b -i Failover.sql

Mirrored Failover

Failover Commands (Failover.sql)---RUN THIS COMMAND TO FAILOVER TO MIRRROR SERVER---

---RUN ON THE MIRROR SERVER TO FAILOVER BACK TO ORIGINAL PRINCIPAL SERVER---

ALTER DATABASE FIMSynchronizationService SET PARTNER FAILOVER

--ALTER DATABASE FIMSynchronizationService SET Force_Service_Data_Allow_Data_Loss

Scripting Failover - Activation

echo C O M P L E T E  P C N S  C U T O V E R

echo ----------------------------------------------------------

echo Failing over the PCNS targets, this process MUST complete prior to running MIISActivate

admod -b "CN=fimprimary,CN=Password Change Notification Service,CN=System,DC=dom,DC=com" "MS-MIIS-PCNS-TargetDisabled::TRUE" -exterr

echo B E G I N  S T A N D B Y S T A R T U P

echo ----------------------------------------------------------

echo Running MIISActivate - prepare to enter the password for the SVC.FIMSYNC account...

"d:\program files\Microsoft Forefront Identity Manager\2010\Synchronization Service\bin\miisactivate.exe" G:\Backups\Keyring\FIMkeyring.bin %USERDNSDOMAIN%\svc.fimsync *

sc \\ilmstandby config FIMSynchronizationService start= auto

 

echo Enabling dormant scheduled tasks

schtasks /Change /TN “FIM-Delta-Loop" /Enable

schtasks /Change /TN "Daily-Maint" /Enable

schtasks /Change /TN "ClearFIMRuns" /Enable

Now for some DFS…

DFS – Crucial Conversations

• Speaking to your AD Admin– Q:“I need a Domain based DFS Namespace called ‘x’ please”– A:”Why do you need a Domain DFS Namespace?”– A:”I need the referral process to be HA and I want to leverage

the domain path instead of a server”– Q:”Wow, you seem to know a lot about DFS, did you attend a

session regarding this at TEC 2010?”

• Asking for replication groups– Q:”I need Delegated Management Permissions to create and

manage replication groups for my DFS namespace please”– A:”I’m sure I saw you in that session at TEC 2010…just make

sure you’re a Local Admin on your replicas”

Data Replication

Left behind but not forgotten

• MAData\ contents• Run Profile Scripts• Custom Tools and Utilities • Dependent libraries in the GAC• Historical reports• Scheduled Tasks• SQL Agent Jobs (non-clustered instance)

Safely in SQL

• Extension libraries• Server configuration• SQL Agent Jobs (cluster only – does not come over in a restore)

Distributed File System

DFS Role in Windows Server 2008• DFS Namespaces• DFS Replication• Same engine that AD uses to replicate data

Major improvements over FRS• Access based enumeration (not enabled by default, 2008 Namespace required)• Failover Cluster support for standalone namespaces• Improved management tools (dfsutil/dfsdiag)

Two types

• Standalone• Domain (Domain DFS in 2000)

Performs client-side load balancing across multiple targets

DFS Referrals Illustrated

AD Site based referral process• Client will auto-select a replica in their own site

Multi-master replication• Differential based (RDC)

How do I advertise my replicated folders?

DFS Namespaces

Windows Server 2008 Mode Domain Based Namespaces

• Forest must be Server 2003 functional level or better• Domain must be Server 2008 functional level or better• DFS Servers must be running Server 2008 or better• Not needed for our purposes, but create namespaces in this mode if you can

Namespaces are a collection of Replication Groups

• Direct server names are never referenced, only the domain (think SYSVOL or Netlogon)More about replication groups…

DFS Replication Groups

Improvements in 2008

• Content Freshness• Improved handling of unexpected shutdowns• Faster replication• Lower network bandwidth utilization• Asynchronous• Higher concurrency• Replicate now feature• Support for RODC’s and Sysvol replication

Pictures!!

DFS Replication Groups

How about a DFS walkthrough?

DFS Setup - Namespace

Most likely installed by a domain admin

• Namespaces do not need to be hosted or maintained on your servers• Install once, then add replication groups• Good to group based on delegation needs (FIM, File Services, etc)

DFS Setup - Namespace

• New Namespace• Browse for a server to

host the namespace

DFS Step-by-Step Guide for Windows Server 2008

DFS Setup - Namespace

DFS Setup - Namespace

DFS Setup - Namespace

DFS Setup – Replication Groups

Delegating Permissions in 2008

DFS Setup – Replication Group

• In the New Replication Group Wizard:– On the Replication Group Type step, select Multi-purpose replication group– On the Name and Domain step, enter the information from the table above– On the Branch Server step, enter the name of primary FIM Server

• NOTE: The primary server will have authority over data replicated to the secondary, thus it is the branch server in this scenario– On the Replicated Folders step, click Add and then Browse to select the first folder to be replicated from the table above

• NOTE: Do not add all folders under the same replication group - this will compromise how folders on the secondary server appear– On the Hub Server step, type the name of the secondary (failover) FIM Server or browse for it– On the Target Folder on Hub Server step, enter the destination root path on the secondary server for the folder to replicate to - you want the Source on Branch

Server and Target on Hub Server to be identical when you are finished– On the Replication Group Schedule and Bandwidth step, select the appropriate option for your deployment - for the greatest resiliency select the Replicate

continuously using the specified bandwidth with the Full bandwidth option selected– Click Create to complete, repeat for each row in from the table above

• Under the Replication node now the new group should appear, select it and click the Replicated Folders tab.• On the Replicated Folders tab, select the replicated folder and select Properties from the Actions pane• On the Properties page for the replicated folder, enter any applicable File Filters or Subfolder Filters from the table above• If the DFS Namespace has been created at this point, the Replication Groups can be published, for each of the groups:

– Select the Replication Group and click the Replicated Folders tab– Select the folder and then click Share and Publish in Namespace option from the Actions pane– In the Share and Publish Replicated Folder wizard:

• On the Publishing Method step, leave the default to Share and publish the replicated folder in a namespace and click Next twice to accept the defaults• On the Namespace Path step, click Browse and select the existing DFS Namespace that was created - click Next to proceed and then Share to complete

Name of Replication Group

Optional description of replication group Domain Replicated Folders File FIlter Subfolder Filter

WF WF Middleware configuration Domain.com D:\WF Default None

FIMTasks FIM run profile automation scripts Domain.com D:\FIMTasks Default None

FIMRootRoot location for FIM configuration file and programs

Domain.comD:\Program Files\Microsoft Forefront Identity Integration Manager\2010

Default Bin, ExtensionsCache, UIShell

Now for some Group Policy!

Group Policy – Security Settings

Computer Configuration\Policies\Windows Settings\Security Settings\Local Policies\User Rights Assignment

• Deny access to this computer from the network domain\svc.fimsync, domain\svc.fimadma• Deny log on as a batch job domain\svc.fimsync, domain\svc.fimadma• Deny log on locally domain\svc.fimsync, domain\svc.fimadma• Deny log on through Terminal Services domain\svc.fimsync, domain\svc.fimadma• Lock pages in memory domain\svc.sql01, domain\svc.sql02• Log on as a batch job BUILTIN\Performance Log Users, domain\svc.fimbatch, BUILTIN\IIS_IUSRS, BUILTIN\Backup Operators, BUILTIN\Administrators

Local Policies\Security Options\Interactive Logon\Restricted Groups

• BUILTIN\Administrators domain\svc.fimsync, domain\Domain Admins, domain\FIM Admins, <localserver>\Administrator

I *heart* GPO Preferences!

Group Policy Preferences

Preferences available in

• 2008+• 2003 and XP SP2 with CSE deployed

Must be edited from

• 2008+• Vista• Windows 7

No requirements on domain or forest functional level!

Group Policy Preferences

Computer Configuration\Preferences\Control Panel Settings\Scheduled Tasks

Item Level Targeting

• Allows for filtering application based on WMI filters

• Many provided by default

• Custom query still available

Questions?

Answers