integration services in the real world brian garraty, database administrator virginia beach city...

INTEGRATION SERVICES IN THE REAL WORLDBrian Garraty, Database Administrator

Virginia Beach City Public Schools

Integration Services in the Real World, Brian Garraty

Who I Am

DBA for Virginia Beach City Public Schools

10 years working with SQL Server 5 years working with SSIS MCITP: DBA


Who You Are

DBAs? Developers? DBAs with Development Background? SSIS experience? DTS experience? SSIS fans? SSIS haters?


Itinerary

Introduction Real World SSIS Case Studies Wrap Up Q&A Resources


Can you farm?” –Mitch Hedberg

“You sure are a great cook!


What SSIS Buys Me

Flexibility Performance Rapid development Dependability Security


SSIS Rather Than…

xp_cmdshell bcp/BULK INSERT osql/SQLCMD Linked Servers Stand-alone .NET Projects


How I Learned

Read “Paradigm Shift” article Rewrote monster, unwieldy stored

procedure True user documentation (blogs) Rewrote first SSIS project


A lot of problems have started looking like nails.” – Me

“Since SSIS has been my hammer,


Storing Snapshot of Active Directory in SQL Server Tables

Case Study #1


AD Snapshot: Requirements

Query Active Directory for: User Accounts Groups Group Memberships

QA the results Load data into SQL Server tables


AD Snapshot: Linked Server Approach

Create linked server to ADSI Query linked server to temp tables QA temp tables (2nd iteration) Replace data in destination tables from

temp table


AD Snapshot: Linked Server Problems

Partial results (limited to 2000 results) Timeouts without error message “An error occurred” Errors Not Configurable


AD Snapshot: SSIS Approach

Script tasks in Control Flow Query AD via

DirectoryServices.DirectorySearcher Stage results in XML

QA Execute SQL Task for row counts Script task to check threshold and throw

error


AD Snapshot: SSIS Approach (con’t)

Expression Precedence Constraint For each item

Truncate via Execute SQL Task Data Flow Task loads table from XML


Index Defragmentation

Case Study #2


Index Defrag: Requirements

Targeted defragmentation Configurable thresholds Support for 2005 syntax and features Report-only mode


Index Defrag: Without SSIS

Many choices Powershell Stored procedure SMO

Valid options - No problems slide here


Index Defrag: SSIS Approach

Generic SSIS package – can run on any server

Store thresholds in configuration ForEach with SMO to loop dbs


Index Defrag: SSIS (con’t)

Dynamic SQL to query indexes sys.dm_db_index_physical_stats sys.dm_db_index_usage_stats

ForEach with NodeList to loop indexes Script Task to build defrag statement


Other Projects

Monster Stored Procedure

Queries by Committee

Rendered Report Compare

Nightly Data Extracts via Secure-FTP


Wrap Up


Praise for SSIS

Parallelism Expressions Configurations .NET


Praise for SSIS (con’t)

Debugging Data Viewers Logging Security

No linked servers Not necessarily dependent on SQL Service

Account


Gripes: Dark Side of SSIS

GUI intensive Visual Studio outside DBA comfort zone Requires .NET Skills Helpful What you see != What you get

Expressions Configurations


Gripes: Dark Side of SSIS (con’t) Source Control Integration Expressions – Syntax and Editor Copy/Paste and ID’s Sorting – DB vs. SSIS


My* Top 10 Tips and Best Practices1. Store packages as files - always2. ROOT_FOLDER and common directory

structure3. Naming Conventions - Project, Package,

and Task4. Use indirect configurations5. Break project into multiple packages


My* Top 10 Tips and Best Practices6. Do not sort in T-SQL7. Use logging – expression can

timestamp file name8. Learn to use ForEach container with

NodeList9. Use CmdExec steps in SQL Agent with

Proxy Accounts to run packages10. Create separate SSMS solution to

manage source control of all T-SQL code


Questions and Answers?


Resources

Jamie Thomson’s old blog (now on SQLBLOG) http://bit.ly/5BeYDh

Paradigm Shift Article on SSC http://bit.ly/5nMIks

SQL Lunch http://sqllunch.com

My Blog http://NULLgarity.wordpress.com

#SSISHELP

http://nullgarity.wordpress.com/

http://nullgarity.wordpress.com/


Brian Garratytwitter.com/@[email protected]

Thanks!

integration services in the real world brian garraty, database administrator virginia beach city...

Documents