report automation in institutional research · report automation in institutional research 18 96....
TRANSCRIPT
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research
Report Automation in Institutional Research
Richard Hetherington
Merima Babic
Courtney Raeford
temple.edu/ira
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 1
Table of Contents
NEAIR 2015 Presentation – Report Automation in Institutional Research ......... 2
Oracle Client Install Instructions ........................................................................................ 14
Visual Basic.NET Code ............................................................................................................. 16
Resources ...................................................................................................................................... 20
Visual Basic.NET ..................................................................................................................... 20
Structured Query Language (SQL) .................................................................................. 20
Excel ............................................................................................................................................. 20
Report Automation in Institutional Research
Rich Hetherington Associate Director
Merima Babic Research Analyst
Courtney Raeford Research Analyst
Data Analysis and Reporting
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Overview
Automation of cyclic reporting VB.net application Excel templates (VLOOKUP) Structured Query Language (SQL)
Process to our solution Demonstration of automated report Adaptability to IR offices
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Located in Philadelphia, Pennsylvania 18 Schools/Colleges 5 Professional Schools
9 Campuses (including Rome & Japan) 450+ Academic Programs 38,000+ Students Approximately 3,500 Faculty www.temple.edu/ira
Temple University At-A-Glance
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Volume of reports Consistency Resources
Goal: More time for
analytical research
Why Report Automation?
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Streamline and Automate Cyclic Reporting while having: Data consistency Process flexibility Self Sufficiency Transparency Scalability
Design Constraints
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Excel Output PDF Output Existing SQL code Ability to schedule Uncomplicated configuration Existing Excel Templates Minimal IT Unit support
Design Constraints
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
VB.net Application
(.EXE)Database
ConnectionSend SQL Request
Set resulting table to variable
Open Excel Template
Go to Worksheet (“Data”)
Paste Data Result
Refresh Report Worksheet
(VLOOKUPS)
Save Excel File Create and Save PDF
Send Notification
Windows Task Scheduler
Report Output
Automation Process
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Excel Template
Automation Process
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Demonstration
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA
Scalability Scheduling Distribution Leverage Resources In-office skills Institutional Resources Online Resources
Efficiency Consistency
Overview
Temple At-A-Glance
Why Automation?
Design Constraints
Automation Process
Demonstration
Conclusion
Questions
Conclusion
Questions?Questions?
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 14
Oracle Client Install Instructions
In order to connect to the oracle environment via Visual Studio, the Oracle Data Access Components (ODAC) must be installed. The version used by the visual basic code provided, can be downloaded from: http://www.oracle.com/technetwork/topics/dotnet/utilsoft-086879.html
Once the ODAC client has been installed, add the reference to your project in Visual Studio. Under
Project, select Add Reference.
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 15
Navigate to the .NET tab and find the Oracle.DataAccess component – double click to add to you
project.
Next, return to the Add Reference dialog box, navigate to the COM tab, find the Oracle Data Control component – double click to add to your project.
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 16
Visual Basic.NET Code
1. Imports System.Text 2. Imports System.IO 3. Imports System.Xml 4. Imports Oracle.DataAccess.Client 5. Imports Oracle 6. Imports System.Net 7. Imports System.Windows.Forms 8. Imports System.Management 9. 'Must add References (.NET, Microsoft.Office.Interop.Excel) 10. Imports Microsoft.Office.Interop.Excel 11. Imports Microsoft.Office.Interop 12. Module Module1 13. 14. Sub Main() 15. Dim uglist1 As New ArrayList 16. Dim uglist2 As New ArrayList 17. 'Reads ini to get path and filename for sql, excel template file, Output filename and
output directory 18. Dim myini As String 19. Dim aPath As String 20. Dim aName As String 21. Dim line4 As String = "" 22. 'Determine the location of the ini file based on exe path 23. aName = System.Reflection.Assembly.GetExecutingAssembly. _ 24. GetModules()(0).FullyQualifiedName 25. aPath = System.IO.Path.GetDirectoryName(aName) 26. myini = aPath & "\Param.ini" 27. 28. Try 29. ' Open the file using a stream reader. 30. Using sr As New StreamReader(myini) 31. line4 = File.ReadAllText(myini) 32. End Using 33. Catch er As Exception 34. End Try 35. 'Parse out the components of the ini file and set to variables 36. Dim myparams As String = line4 37. Dim mypparts As String() = myparams.Split(";") 38. Dim mysqlin As String = mypparts(0) 39. Dim myxls As String = mypparts(1) 40. Dim myoutputfolder As String = mypparts(2) 41. Dim myfilename As String = mypparts(3) 42. Dim line As String = "" 43. '************************************** 44. Try 45. ' Open the sql file using a stream reader.
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 17
46. Using sr As New StreamReader(mysqlin) 47. line = sr.ReadToEnd() 48. End Using 49. Catch er As Exception 50. 'Console.WriteLine("The file could not be read:") 51. End Try 52. '************************************** 53. '**** OPEN ORACLE CONNECTION ********* 54. '************************************** 55. Dim orapass As String = My.Resources.ORA_PASS 56. Dim oradb As String = "Data Source=(DESCRIPTION=" _ 57. + "(ADDRESS=(PROTOCOL=TCP)(HOST=prd-rac1.erp.temple.edu)(PORT=1521))" _ 58. + "(CONNECT_DATA=(SERVICE_NAME=PROD8)));" _ 59. + "User Id= ADD_ORACLE_USERNAME;Password= " & orapass & ";" 60. '************************************** 61. Dim conno As New OracleConnection(oradb) 62. conno.Open() 63. Dim sqlo As String 64. Dim cmdo As New OracleCommand 65. Dim slbcount As Integer 66. '****** check for connection *********** 67. If (conno.State = 0) Then 68. MsgBox("Oracle Connection Failure") 69. Else 70. 'MsgBox("open") 71. End If 72. '*************************************** 73. sqlo = "" 74. sqlo = line 75. cmdo.CommandType = CommandType.Text 76. cmdo.CommandText = sqlo 77. cmdo.Connection = conno 78. Dim dro As OracleDataReader = cmdo.ExecuteReader() 79. 'Passes sql into oracle connection - waits for returning data 80. While dro.Read() 81. If dro.HasRows = False Then 82. MsgBox("No Data Selected") 83. GoTo SKIPifblank 84. Else 85. uglist1.Add(dro.Item("METRIC")) 86. uglist2.Add(dro.Item("VALUE")) 87. Try 88. Catch ex As Exception 89. 'Console.WriteLine(ex.Message) 90. Return 91. End Try 92. End If 93. skipifblank: 94. End While 95. dro.Dispose()
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 18
96. eoj: 97. slbcount = uglist1.Count 98. cmdo.Dispose() 99. conno.Dispose() 100. 'Load sql output to excel template file 101. Dim xlApp As New Excel.Application 102. Dim xlWorkBook As Excel.Workbook 103. Dim xlWorkSheet As Excel.Worksheet 104. Dim xlSourceRange As Excel.Range 105. Dim xlDestRange1 As Excel.Range 106. Dim xlDestRange2 As Excel.Range 107. Dim myxlfile As String 108. myxlfile = myxls 109. xlWorkBook = xlApp.Workbooks.Open(myxlfile) 110. 'Display Excel 111. xlApp.Visible = True 112. 'Set the source worksheet 113. xlWorkSheet = xlWorkBook.Sheets("Data") 114. 'Set the source range 115. xlSourceRange = xlWorkSheet.Range("A2:B455") 116. 'Clear current data 117. xlSourceRange.Delete() 118. 'Set the destination range 119. xlDestRange1 = xlWorkSheet.Range("A2:A455") 120. xlDestRange1.Value =
xlApp.Application.WorksheetFunction.Transpose(uglist1.ToArray) 121. xlDestRange2 = xlWorkSheet.Range("B2:B455") 122. xlDestRange2.Value =
xlApp.Application.WorksheetFunction.Transpose(uglist2.ToArray) 123. xlWorkSheet = xlWorkBook.Sheets("REPORT") 124. xlWorkSheet.Activate() 125. Dim mydatestamp As String = DateTime.Now.ToString("yyyy-M-d") 126. Dim myexceltitle As String = myfilename & "_" & mydatestamp & ".xlsx" 127. xlWorkSheet.SaveAs(myoutputfolder & myexceltitle) 128. ' xlWorkSheet.ExportAsFixedFormat(XlFixedFormatType.xlTypePDF, myoutputfolder
& myexceltitle, Excel.XlFixedFormatQuality.xlQualityStandard, True, False, 1, 10, False)
129. xlWorkSheet.ExportAsFixedFormat(XlFixedFormatType.xlTypePDF, myoutputfolder & myfilename & "_" & mydatestamp, Excel.XlFixedFormatQuality.xlQualityStandard, True, False, 1, 10, False)
130. 131. xlWorkBook.Close() 132. xlApp.Quit() 133. 134. ReleaseObject(xlDestRange1) 135. ReleaseObject(xlDestRange2) 136. ReleaseObject(xlWorkSheet) 137. ReleaseObject(xlWorkBook) 138. ReleaseObject(xlApp) 139. ' MsgBox("Process Complete")
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 19
140. End Sub 141. 142. Private Sub ReleaseObject(ByVal obj As Object) 143. 'This function is required in order to close all files and connections 144. Try 145. Dim intRel As Integer = 0 146. Do 147. intRel = System.Runtime.InteropServices.Marshal.ReleaseComObject(obj) 148. Loop While intRel > 0 149. 'MsgBox("Final Released obj # " & intRel) 150. Catch ex As Exception 151. 'MsgBox("Error releasing object" & ex.ToString) 152. obj = Nothing 153. Finally 154. GC.Collect() 155. End Try 156. End Sub 157. End Module
NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 20
Resources
Visual Basic.NET For more information about connecting to an oracle database environment using Visual Studio: http://www.oracle.com/webfolder/technetwork/tutorials/obe/db/dotnet/GettingStartedVBVersion/GettingStartedNET_VBVersion.htm For more information about How to automate Microsoft Excel from Visual Basic .NET: https://support.microsoft.com/en-us/kb/301982 Visual Basic Quick Syntax Reference: http://www.digilife.be/quickreferences/QRC/VB.NET%20Quick%20Reference.pdf Microsoft’s Visual Basic Programming Guide: https://msdn.microsoft.com/en-us/library/y4wf33f0(v=vs.100).aspx
Structured Query Language (SQL) Oracle SQL Online Language Guide: https://docs.oracle.com/cloud/latest/db112/SQLRF/toc.htm For Microsoft SQL Server environments, the following resources can assist with connection strings and syntax: https://www.connectionstrings.com/sql-server/ https://support.microsoft.com/en-us/kb/308656
Excel VLOOKUP Reference: https://support.office.com/en-us/article/VLOOKUP-function-0bbc8083-26fe-4963-8ab8-93a18ad188a1