Employee Time Analysis Data Load Process

(SSIS) 2005

***

John V. Stires

October 10, 2009

Page | 1

Table of ContentsI. Introduction..............................................................................................................................3

II. Purpose.................................................................................................................................3

III. Overview...............................................................................................................................3

A. Counter Variables.............................................................................................................3

B. Process Control.................................................................................................................3

IV. Package Detail......................................................................................................................5

A. Employee Master Upload..................................................................................................5

B. Employee Rates Upload....................................................................................................7

C. Division Master Upload....................................................................................................9

D. Client Master Upload......................................................................................................11

E. Client Grouping Master Upload.........................................................................................13

F. Client Groupings Client Xref Upload.................................................................................15

G. Project Master Upload....................................................................................................17

H. Time Sheet Upload..........................................................................................................19

I. DB Maintenance.................................................................................................................22

Page | 2

I. IntroductionDesign and build a SQL Server 2005 database to track employee and customer information, timesheet and labor rates data, as well as job order information, job materials, and customer invoices for the fictitious company, Allworks, Inc. AllWorks currently stores this information in Excel Spreadsheets, XML files, and csv files.

II. PurposeThis project is designed to load data from various sources into an uniform SQL 2005 Server database. Eight tables are loaded with the external data sources each of these tables will be uploaded by a separate package.

III. OverviewThere are eight packages that upload the data from various external sources. Every attempt was made to avoid abortive errors because of data issues. However, when such an event does occur, an email is sent and a message written to an error log giving notification that an error has occurred. The packages and tables addressed here are:

Table Upload PackageEmployee Master Upload EmployeesEmployee Rates Upload EmployeeRatesDivision Master Upload DivisionClient Master Upload ClientsClient Grouping Master Upload ClientGroupingsClient Groupings Client Xref Upload ClientGroupingsXClientProject Master Upload JobMasterTime Sheet Upload JobTimeSheets

There are two remaining packages. There is a package, DB Maintenance, which performs maintenance on the database immediately after the successful completion of the uploading process. The other package, Master Payroll Package Control, coordinates the running of all of the other packages. Each of these packages will be discussed further in detail below.

A. Counter Variables1. Total Input rows- TtlRows2. Changed Rows - ChangedRows3. Unchanged Rows - UnchangedRows4. Rows in Err - ErrorRows

B. Process ControlThis process control drives the entire upload process. This package coordinates the execution of all of the other packages to perform the data upload. Upon successful completion or should an error occur, it sends an email.

Page | 3

Process Control: This process control drives the entire upload process.

Page | 4

IV. Package DetailEach package performs its upload. When successful, it sends an email notification of that fact giving the statistics for the run in terms of the number of new rows, updated rows, etc. There is a standard naming for the counters across all packages. Others will be discussed in those packages where they are used:

A. Employee Master Upload1. Input

Excel spreadsheet file -Employees.XLS, Sheet: Employees

2. OutputThe Employees table in the AllWorksDBStudent database.

3. Special Processing.There is a new, derived, field created for FullName.

4. Lookups / ValidationsA check is made to verify whether this employee is already in the table or not.

5. Conditional Processing.a) The conditions processed are:b) New Rows are added to the table.c) Updated rows where rows are modifiedd) Error rows. Errors include input data errors where the input Employee ID

is null.6. Package Diagram

Page | 5

Employee Upload: This process. Uploads new Employee Data.

Page | 6

B. Employee Rates Upload1. Input

Excel spreadsheet file - Employees.XLS, Sheet: EmployeeRates

2. OutputThe EmployeeRates table in the AllWorksDBStudent database.

3. Special Processing.N/A

4. Lookups / Validationsa) Verify if this employee is in the Employees table.b) Determine if there is already a row for this employee in the EmployeeRates

table.5. Conditional Processing.

The conditions processed are:a) New Rows are added to the table.b) Updated rows where rows are modifiedc) Unchanged rows that were not modifiedd) Error rows. Errors include input data errors where the input Employee ID

is null6. Package Diagram

Page | 7

Employee Rates Upload: This process uploads the Employee Rates Data.

Page | 8

C. Division Master Upload1. Input

Excel spreadsheet file - ClientGeographies.XLS, Sheet: DivisionDefinitions

2. OutputThe Division table in the AllWorksDBStudent database.

3. Special Processing.N/A

4. Lookups / Validationsa) The Division table in the AllWorksDBStudent database.b) Determine if there is already a rows for this division in the Division table.

5. Conditional Processing.The conditions processed are:a) New Rows are added to the table.b) Updated rows where rows are modifiedc) Unchanged rows that were not modifiedd) Error rows. Errors include input data errors where the input Employee ID

is null6. Package Diagram

Page | 9

7. Division Upload: This process uploads new Division Data.

Page | 10

D. Client Master Upload1. Input

Excel spreadsheet file - ClientGeopgraphies.XLS, Sheet: Client Listing

2. OutputThe Client table in the AllWorksDBStudent database.

3. Special Processing.N/A

4. Lookups / ValidationsA check is made to verify whether this client is already in the table or not.

5. Conditional Processing.The conditions processed are:a) New Rows are added to the table.b) Updated rows where rows are modifiedc) Unchanged rows that were not modifiedd) Error rows. Errors include input data errors where the input Employee ID

is null6. Package Diagram

Page | 11

Client Master Upload: This process uploads new and changed Client Data.

Page | 12

E. Client Grouping Master Upload1. Input

Excel spreadsheet file - ClientGeopgraphies.XLS, Sheet: Special Groupings

2. OutputThe ClientGroupings table in the AllWorksDBStudent database.

3. Special Processing.Duplicate input records are expected and all duplicates are eliminated from the data run stream. A duplicate is where there are input records with both the Client Grouping Id and the Grouping Name the same.

4. Lookups / ValidationsA check is made to verify whether this client is already in the table or not.

5. Conditional Processing.The conditions processed are:a) New Rows are added to the table.b) Updated rows where rows are modifiedc) Unchanged rows that were not modifiedd) Error rows. Errors include input data errors where the input Employee ID

is null6. Package Diagram

Page | 13

Client Groupings Master Upload: This process uploads new Client Groupings Data.

Page | 14

F. Client Groupings Client Xref Upload1. Input

Excel spreadsheet file - ClientGeopgraphies.XLS, Sheet: Special Groupings

2. OutputThe ClientGroupingsXClients table in the AllWorksDBStudent database.

3. Special Processing.N/A.

4. Lookups / Validationsa) Verify the Client is in the Clients table.b) Verify if this Client Grouping ID is in the Client Groupings table.c) Determine if there is already a row for this ClientGroupings Xref Clients

record in the ClientGroupingsXClients table.5. Conditional Processing.

The conditions processed are:a) New Rows are added to the table.b) Error rows. Errors include input data errors where the input Client

Grouping ID, Client Grouping ID is null or the Record entry is already in the table.

6. Package Diagram

Page | 15

Client Groupings Xref Upload: This process uploads new Client Groupings Cross Reference Data.

Page | 16

G. Project Master Upload1. Input

Excel spreadsheet file - ProjectMaster.XLS, Sheet: ProjectMaster

2. OutputThe JobMaster table in the AllWorksDBStudent database.

3. Special Processing.N/A.

1. Lookups / Validationsa) Verify the client ID is in the Clients tableb) Determine if the project is already in the JobMaster

4. Conditional Processing.The conditions processed are:a) New Rows are added to the table.b) Updated rows where rows are modifiedc) Unchanged rows that were not modifiedd) Error rows. Errors include input data errors where the input Project ID or

Client ID are null, or the client ID does not exist in the Clients table.5. Package Diagram

Page | 17

Project Master Upload: This process uploads new Project Master Data.

Page | 18

H. Time Sheet Upload1. Input

Excel spreadsheet file - EmpTime*.CSV2. Output

The JobTimeSheets table in the AllWorksDBStudent database.3. Special Processing.

There are a number file input files to be processed. The file name pattern is: EmpTime*.CSV. The variable TimeSheetFile is used to process each file through the load.

4. Lookups / Validationsa) Verify the Employee ID is in the Employee tableb) Verify the Project ID is in the JobMaster tablec) Determine if the Job Time Sheet is already in the JobTimeSheets tables

5. Conditional Processing.The conditions processed are:a) New Rows are added to the table.b) Updated rows where rows are modifiedc) Unchanged records are not modified.d) Time Sheets for job not in the JobMastere) Time Sheets with WorkDates after the job Closed Date, Late Time Sheetsf) Error rows. Errors include input data errors where:

1. Input Employee ID or Project ID or WorkDate are null2. The Employee ID does not exist in the Employees table 3. Project ID does not exist in the JobMaster table.

6. Package Diagram

Page | 19

a) Control Flow

Time Sheet Upload: This process uploads new Time Sheet Data.

Page | 20

b) Data Flow

Page | 21

I. DB MaintenanceThis package handles the duties of performing maintenance on the AllWorksDBStudent database. The functions it performs are:1. Backing up the database2. Rebuilding the indexes3. Shrinking the database to eliminate unused space4. Sending an email upon a successful completion5. Package Diagram

Page | 22

Database Maintenance

Page | 23