Database Continuous Integration, Unit Test and Functional Test

About myself:

• Data architect/DBA with 16 years of SQL Server experience

• Independent consultant, currently Architecture Lead and Technical Delivery Manager at Government of Alberta

• Microsoft Certified System Engineer: MCSE

• Oracle Certified Professional: Oracle DBA

• IBM Certified Solution Expert: DB2 UDB

http://netdbsolutions.comhttps://twitter.com/HarryZheng

http://ca.linkedin.com/in/harryzhenghttps://www.facebook.com/Harry.H.Zheng

2

Session agenda

1. What is Continuous Integration?

2. Database Continuous Integration Setup

3. Database unit test and functional test

4. Database test standard and best practice

5. Tips & Tricks on writing database tests

6. Q&A

3

What is Continuous Integration

• Continuous integration (CI) – the process of continuously integrating developers code in order to find problems quickly

• Wikipedia: CI is the practice of merging all developer working copies with a shared mainline several times a day.

• CI is often related to Test Driven Development (TDD) and Agile practice

4

Traditional work flow

1. Dev A works on his computer and checks in code when ready

2. Dev B works on his computer and checks in code when his task is done

3. Dev C…4. When time to release, we start up build

process and see if the combined code works

5. The version of code doesn’t build6. Who is responsible?

5

Continuous Integration Work Flow

6

What does CI check?

• Does the solution compile?• Does the code pass our unit tests?• Does the code meet functional

requirements?• Does the version meet code quality

requirements?

7

What does CI produce?

• Automatically produce build artifacts• Ensure same version can be staged to

various environments• Setup for automated deployment

8

Benefits of CI

• Ensure code in source control works• Less time on integration• Reduce friction within project team• Save time on build and deploy• Increased visibility of progress• Increased overall confidence on code

9

Do you need CI: scenario 1

• Harry works with 3 other Devs on a side project• Project is a web based app with SQL backend• Harry is the only dba on the team• SQL database is relatively simple with 30 tables and a few

SPs• Do we need setup CI?

10

Do you need CI: scenario 2

• Harry works in a team with 20 developers, 4 DBAs• Main SQL Server has 200+ tables and many SPs, functions• Some developers have good knowledge of SQL skills, others

not so good but can code T-SQL• We used to have a rule that only DBAs can check in SQL

code, but DBAs became a bottom neck quickly• Now we have opened database projects to developers to

check in code changes, bug fixes, etc.• Do we need setup CI?

11

Software for Running CI

• Team City• Cruise Control• Final Builder

12

TeamCity Web Interface

13

Final Builder Server Web Interface

14

Session agenda

1. What is Continuous Integration?

2. Database Continuous Integration Setup

3. Database unit test and functional test

4. Database test standard and best practice

5. Tips & Tricks on writing database tests

6. Q&A

15

Prerequisite for Continuous Integration

• Source Control System• Build Server• Manage Database project with SSDT• Database Tests

16

Database CI Work flow

1. Developer check in code to source control2. Check in triggers a build of the project3. On successful build, the version of the code

is deployed to testing databases4. Unit tests are executed against test

database5. Functional tests are executed against

functional test database6. If all steps succeed, the version is ready for

deployment7. If any step fails, alert is sent to the team

and the cycle starts from step1 again17

Database CI

18

Database deploy script

19

Substitution/Build configuration file<Environment value="BuildDB_CI" machine="VM-DBServer-1" substitute="false"> <Database environmentsuffix="_CI" BackupFileFolder="\\VM-DBServer-1\Deploy"> <SQLInstances> <SQLInstance name="SQLInst1" host="VM-DBServer-1" sqlinstance="\Dev"

databaseFolder="d:\SQLData\Dev" databaseLogFolder="d:\SQLLog\Dev" dbdeployfolder="D:\Deploy"> <Projects> <Project name="DBProj1" seedserver="VM-DBServer-1\Seed" seeddatabase="DB1_Seed" fullbuild="No"> <Databases> <Database name="DBProj1" targetdatabase="DB1_CI" /> </Databases> </Project> <Project name="DBProj2" seedserver="VM-DBServer-1\Seed" seeddatabase="DB2_Seed" fullbuild="No"> <Databases> <Database name="DBProj2" targetdatabase="DB2_CI" /> </Databases> </Project> </Projects> </SQLInstance> </SQLInstances> </Database> <SSIS ServerName="VM-DBServer-1" /> <SSRS ServerName="VM-DBServer-1" ReportServerPath="ReportServer_Dev"/></Environment>

20

Deploy DB schema with SQLPackage

"C:\Program Files (x86)\Microsoft SQL Server\110\DAC\bin\SqlPackage.exe" /Action:Publish /SourceFile:"%1" /TargetConnectionString:"Server=%2;Initial Catalog=%3;Integrated Security=True" /p:BlockOnPossibleDataLoss=False /p:CreateNewDatabase=False /p:DisableAndReenableDdlTriggers=True /p:DropConstraintsNotInSource=True /p:DropDmlTriggersNotInSource=True

21

Deploy database and preserve data

1. Run Pre-deployment scripts2. Execute sqlpackager.exe to deploy

database schema changes3. Run Post-deployment scripts

22

Deploy database and preserve dataWhen to use Pre and Post scripts?•If you change table definition that may cause data loss when database project is deployed, you need to write a pre-deployment script to preserve the data•All other data change scripts will go to Post-deployment script•All Pre and Post scripts should be created so that the scripts can be executed repeatedly

23

Session agenda

1. What is Continuous Integration?

2. Database Continuous Integration Setup

3. Database unit test and functional test

4. Database test standard and best practice

5. Tips & Tricks on writing database tests

6. Q&A

24

What is unit test?

• Test on a discrete unit of code• Unit test should not be affected by other

units• Isolates the unit under test from the rest

of the code• Repeatable

25

My definition of database unit test

• Test that runs quickly, and does not rely on existing data with the exception of static data (e.g.., reference data)

26

Unit Testing - AAA

•Arrange• Setup data (with TestHelper)

•Act• Call Precedure/Function

•Assert• Compare results with expectation

27

Create Unit Test From SSOE from VS

28

Unit Test Example for SP

CREATE PROCEDURE [dbo].[uspShowMeetingAttendee]@MeetingID int = 0

ASSELECT MT.Name,M.FirstName,M.[LastName]From Meeting MTinner join MeetingAttendee MAon MT.MeetingID = MA.MeetingIDinner join Member Mon M.MemberID = MA.MemberID

RETURN 0

29

Unit Test Example-- database unit test for dbo.uspShowMeetingAttendeeBEGIN TRANSACTION

--ArrangeDeclare @MeetingID int

INSERT INTO [dbo].[Meeting] ([Name]) VALUES ('XXTestMeeting')

select @MeetingID = MeetingID from Meeting where Name = 'XXTestMeeting'

INSERT INTO [dbo].[Member]([FirstName],[LastName],[BirthDate]) VALUES ('XXFirstName','XXLastName','1970-01-01')

INSERT INTO [dbo].[MeetingAttendee] ([MeetingID],[MemberID]) Select MeetingID, MemberID From Meeting M1 inner Join Member M2 on 1=1 where M1.Name = 'XXTestMeeting' and M2.LastName = 'XXLastName'

--code to create a dynamic temporary table ##xx{Schema}{SPName}xx -- based of the output of a stored procedure exec [TestHelper].[uspCreateTableForStoredProcedureOutput]@storedProcedure = '[dbo].[uspShowMeetingAttendee]'

--ACT--insert into the temporary table created by the SPinsert into ##xxdbouspShowMeetingAttendeexxEXECUTE [dbo].[uspShowMeetingAttendee] @MeetingID = @MeetingID;

--ASSERT--now you can select specific column without knowledge of the oridinal valueselect LastNamefrom ##xxdbouspShowMeetingAttendeexx where FirstName = 'XXFirstName'

ROLLBACK TRANSACTION

30

What is functional test?

• Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test

• Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered

31

My definition of database functional test

• Tests with long execution times and they rely on existing non-static data

• Functional tests will run on a server where the complete dataset is available.

32

Database Test Tools and Frameworks

• Microsoft SSDT • http://msdn.microsoft.com/en-ca/data/tools.aspx

• TSQLT • http://tsqlt.org/

• Red-Gate SQL Test• http://www.red-gate.com/products/sql-development/sql-test/

33

Session agenda

1. What is Continuous Integration?

2. Database Continuous Integration Setup

3. Database unit test and functional test

4. Database test standard and best practice

5. Tips & Tricks on writing database tests

6. Q&A

34

Database test standard and best practice

• The folder structure within the database test project will have a structure that closely resembles the structure of the database project.

35

Naming Conventions

 

Database tests must be discoverable. All database tests will follow one of these naming conventions: <Database Object Name>_Main_<Description>.cs<Database Object Name>_State_<Description>.cs<Database Object Name>_Defect_<Defect #>.cs36

Rules

 

• All database objects must have at least one main test.

• Static tables (e.g., reference data) must have one or more state tests.

• A defect that results in a change to a database object must have at least one defect test.

 

37

Rules

 

• All test scripts must not change the state of the database.

• This is done by adding a Begin Tran/ Rollback Tran code block to the test script.  

38

Test Helpers

• In order to assist in creating database tests, the database has a number of stored procedures to create test data.

• All of these stored procedures exist in the [TestHelper] schema.

• Developers are free to modify existing [TestHelper] stored procedures provided the changes do not alter the original purpose of the stored procedure and the changes do not impact any existing database tests.

 39

Documenting Tests

All test scripts should be well documented. 1.The comment will indicate which test condition(s) are associated with a particular result set.

2.The comments will indicate what part of the SQL in the database object is being exercised (e.g., which case statements within a where clause are exercised).  

40

Session agenda

1. What is Continuous Integration?

2. Database Continuous Integration Setup

3. Database unit test and functional test

4. Database test standard and best practice

5. Tips & Tricks on writing database tests

6. Q&A

41

Writing tests for SP or Function

 

Problem:•A test that calls to the stored procedure returns a result set with many rows and the test involves retrieving one row only

•A test is only concerned about testing a single value, but the stored procedure returns a result set with many columns

 

42

Writing tests for SP or Function

There are a couple of ways of making the tests less susceptible to changes:  1.Using a dynamic temporary table2.Using a declared temporary table The first is fair more desirable but cannot be used for stored procedure or functions that have dynamic SQL.

 

43

Insert into a dynamic temporary table 

This approach uses sys.dm_exec_decribe_first_result_set_for_object to dynamically create a temporary table that has the same columns as the first result set of the stored procedure.

[TestHelper].[uspCreateTableForStoredProcedureColumns] creates a global temporary table that can be used to store the results of executing the stored procedure under test.  

44

Insert into a dynamic temporary table 

Begin Tran

--code to create a dynamic temporary table ##xx{Schema}{SPName}xx

-- based of the output of a stored procedure  exec [TestHelper].[uspCreateTableForStoredProcedureColumns]

@storedProcedure = '[dbo].[uspRetrieveUserInfo]'

--insert into the temporary table created by the SPinsert into ##xxdbouspRetrieveUserInfoxx

  exec [dbo].[uspRetrieveUserInfo] @UserID = 51 

--now you can select specific column without knowledge of the oridinal value

select FirstName from ##xxdbouspRetrieveUserInfoxx where [BirthDate] > '1995-01-01'

 Rollback Tran 

45

Insert into a dynamic temporary table Create procedure [TestHelper].[uspCreateTableForStoredProcedureOutput] ( @storedProcedure varchar(128))asbegin declare @sql nvarchar(max) = '', @tablename varchar(100) = '##xx' + replace(replace(replace(@storedProcedure,'[',''),']',''),'.','') + 'xx'  set @sql = @sql + 'IF OBJECT_ID(''tempdb..' + @tablename + '''' + ') IS NOT NULL DROP TABLE ' + @tablename + ';' set @sql = @sql + 'create table ' + @tablename + '('  select @sql = @sql + '[' + name + '] ' + coalesce(system_type_name, (D.Data_Type +

case when Character_Maximum_Length is not null then '(' + cast(Character_Maximum_Length as varchar) + ')'

else '' End)) + ',' from sys.dm_exec_describe_first_result_set_for_object (object_id(@storedProcedure), null) FR Left Outer Join INFORMATION_SCHEMA.DOMAINS D on FR.User_Type_Name = D.Domain_Name Order by FR.Column_Ordinal  set @sql = substring(@sql, 0, len(@sql)) + ');' EXEC sp_ExecuteSQL @sql;end

46

Insert into a dynamic temporary table Pros:• Test is not dependent on the order of the

columns in the stored procedure, i.e., the order can change

• Result set can be further filteredCons:• Will not work for stored procedures that

use dynamic SQL • Will not work for stored procedures that

return more that one result set 

47

Insert into a declared temporary table

Declare a temporary table that has the same columns as the result set of the stored procedure.

This is the less desirable of the two options but is the only choice if the stored procedure or function returns the result set using dynamic SQL.  

48

Insert into a declared temporary table Begin Tran 

-- TestHelper SP creates a simple user with a first name of xxFoobarxxdeclare @UserID Intexec [TestHelper].[uspCreateUser] @id = @UserID output

declare @User table ( ID Int, FirstName varchar(50),LastName varchar(50),BirthDate date)

 --Insert the SP output to the temp tableinsert into @User exec dbo.uspSearchUserByName @FirstName = 'xxFoobarxx‘

--this should return a single user--Verify_One_User_Existsselect * from @User where FirstName = 'xxFoobarxx‘

 Rollback Tran  

49

Insert into a declared temporary tablePros:• Further filtering of the result set is

possible. 

Cons:• The order of the columns in the declared

temporary table must be the same as the order of the result set returned by the stored procedure or function.

50

Session agenda

1. What is Continuous Integration?

2. Database Continuous Integration Setup

3. Database unit test and functional test

4. Database test standard and best practice

5. Tips & Tricks on writing database tests

6. Q&A

51

52

Reference and further reading:

1. http://www.pluralsight.com/courses/continuous-integration2. http://en.wikipedia.org/wiki/Continuous_integration3. http://www.rackspace.com/blog/the-business-advantages-

of-continuous-integration/4. http://www.youtube.com/watch?v=PbVKgoAVZjY

(Continuous Integration with the Database by Ike Ellis)5. http://indragunawan.com/2013/09/teamcity-continuous-

integration-for-everybody.html6. http://www.youtube.com/watch?v=4sANX9AhM8c

(Introduction to Continuous Integration)

53