data warehousing at the university at albany wendell g. lorang director of institutional research...
Post on 15-Jan-2016
244 views
TRANSCRIPT
Data Warehousing Data Warehousing at The University at at The University at
AlbanyAlbany
Wendell G. LorangWendell G. Lorang
Director of Institutional ResearchDirector of Institutional Research
University at Albany, SUNYUniversity at Albany, SUNY
AIRPO Winter 2006 ConferenceAIRPO Winter 2006 Conference
Albany, New YorkAlbany, New York
UAlbany implemented a UAlbany implemented a Student Records data Student Records data
warehouse that denormalizes warehouse that denormalizes the data from over 60 the data from over 60
PeopleSoft tables into seven PeopleSoft tables into seven tables. More than 100 queries tables. More than 100 queries
are available to departments to are available to departments to use to obtain the information use to obtain the information
they need. they need.
OverviewOverview
• Our PeopleSoft Our PeopleSoft Implementation Implementation
• Problem DescriptionProblem Description
• The SolutionThe Solution
• The IssuesThe Issues
PresentationPresentation
4
PeopleSoftPeopleSoft at UAlbany
19961996
March 1999March 1999
Project BeganProject Began
Undergrad ProspectsUndergrad Prospects
June 2000June 2000 Human ResourcesHuman Resources
November 2000November 2000 Undergraduate andUndergraduate and
Graduate Graduate AdmissionsAdmissions
March 2001March 2001 Orientation andOrientation and
Summer PlanningSummer Planning
ConferencesConferences
5
PeopleSoftPeopleSoft at UAlbany
June 2003June 2003 Student Records Student Records
January 2004January 2004 Financial AidFinancial Aid
May 2005May 2005 Full Student Full Student FinancialsFinancials
““Can I get a list of all of the grad Can I get a list of all of the grad
students in my department who students in my department who
are currently registered?”are currently registered?”
The Problem – A Common Question
The Problem – A Common QuestionWhat data fields are needed?What data fields are needed?
Term, Empl ID, Name, Career, Term, Empl ID, Name, Career, Department, Program, Registered Department, Program, Registered Credits, Full/Part Time, Admit Term, Credits, Full/Part Time, Admit Term, Statute of Limitations Term, Statute of Limitations Term, Advisor, Last Term Registered, Total Advisor, Last Term Registered, Total CreditsCredits
The Solution - Using Query Tool
• To get the 13 fields requires To get the 13 fields requires linking to 9 tables.linking to 9 tables.
•Logic to find the most recent Logic to find the most recent program/plan that has not been program/plan that has not been discontinued is too complicated for discontinued is too complicated for query toolquery tool
Criteria
Generated SQL
• Too many PeopleSoft tables andToo many PeopleSoft tables and
users don’t know table structureusers don’t know table structure
• Need to join many tables to get Need to join many tables to get
datadata
• Queries run slow against tables Queries run slow against tables
because of joins and security treebecause of joins and security tree
The Problem
What Are We Going to DO???
Write Write SQRsSQRs
??
Write Write QueriesQueries
??
Buy a Buy a reportinreportin
g g productproduct
??
13
THE SOLUTIONTHE SOLUTION
A CUSTOMA CUSTOM
UNIVERSITY AT UNIVERSITY AT ALBANY ALBANY
DATA WAREHOUSEDATA WAREHOUSE
• Primary goals Primary goals
User Access User Access
Answer many varied Answer many varied questions/needsquestions/needs
Real-time management/monitoring Real-time management/monitoring tooltool
The Solution –
Student Data Warehouse
Design and Implementation
• Design led by Institutional Design led by Institutional ResearchResearch
• Met with departments and Met with departments and worked with programmer to define worked with programmer to define the table organizationthe table organization
• Defined the fields to be includedDefined the fields to be included
• Defined the population of Defined the population of students to be includedstudents to be included
• IR testedIR tested
•Data Data NOTNOT included: included:• Scores (ACT, SAT, high school & Scores (ACT, SAT, high school &
transfer GPAs, GRE, etc)transfer GPAs, GRE, etc)• Student Billing dataStudent Billing data• Financial AidFinancial Aid• Racial/ethnic dataRacial/ethnic data• Disability informationDisability information• Student’s SSNStudent’s SSN
The Solution –
Student Data Warehouse
● ● What students to include?What students to include?
● ● How many semesters to How many semesters to include?include?
• How to define registered?How to define registered?
• How to handle addresses?How to handle addresses?
The Issues
What students to include?What students to include? Any student who was registered Any student who was registered or got a grade (even a W or Z) is or got a grade (even a W or Z) is included. included.
In addition, all active (eligible to In addition, all active (eligible to register) students are included.register) students are included.
The Issues
What semesters to include?What semesters to include?
● ● CurrentCurrent
● ● Future (for Advance Future (for Advance Registration)Registration)
●● Past (comparative purposes)Past (comparative purposes)
The Issues
How to Define Registered?How to Define Registered?
a)a)Registered Flag showing that a Registered Flag showing that a student was registered at some student was registered at some point in the semester but point in the semester but subsequently withdrew (with a subsequently withdrew (with a grade) from all courses grade) from all courses
b)b)Presently Registered Flag Presently Registered Flag indicates that the student is indicates that the student is currently (as of the day before) currently (as of the day before) registeredregistered
The Issues
How to Handle Addresses?
Developed concept of Priority Address -- the student’s ‘closest’ address
If a student lives on campus, that is their Priority Address; if not on campus, but locally, that is the Priority Address; otherwise, their permanent address is their Priority Address.
The Issues
Last Enrollment Term
Deceased Students
Managing Size of the Data Warehouse
FERPA
The Issues
• Active students for past and Active students for past and
future semestersfuture semesters•Academic program informationAcademic program information• Student and Course EnrollmentsStudent and Course Enrollments•Course informationCourse information
The Solution –
Student Data Warehouse
• data on majors (demographics, data on majors (demographics,
academic performance, registered academic performance, registered
vs not-registered, etc.)vs not-registered, etc.)•Demographic and academic Demographic and academic
characteristics of students in characteristics of students in
classesclasses• Output to prepare labelsOutput to prepare labels
The Solution –
A Student Data Warehouse
25
•Data from more than 60 PeopleSoft Data from more than 60 PeopleSoft
tables are placed in seven tablestables are placed in seven tables• Tables reside in the same instance Tables reside in the same instance
as our Production databaseas our Production database• Queries are quick – no Queries are quick – no
performance or response time performance or response time
issuesissues
The Solution –
Student Data Warehouse
26
• Refreshed daily based on Refreshed daily based on
parametersparameters• Parameters (terms to be included) Parameters (terms to be included)
can be reset each daycan be reset each day• DW currently includes Fall 2003 DW currently includes Fall 2003
through Fall 2006through Fall 2006
The Solution –
Student Data Warehouse
Queries
• Over 200 public queries written by Over 200 public queries written by IRIR
• 90 queries primarily for academic 90 queries primarily for academic departmentsdepartments
• Return data elements depending on Return data elements depending on requirementsrequirements
• Prompts for fields such as term, Prompts for fields such as term, department, plan, advisor, coursedepartment, plan, advisor, course
• Can link to other PeopleSoft tablesCan link to other PeopleSoft tables
Queries
• Most academic departments have Most academic departments have “run only” query security“run only” query security
• Most changes to existing queries or Most changes to existing queries or new queries are handled by IRnew queries are handled by IR
Documentation
• What is the Data Warehouse? What is the Data Warehouse?
• Terms in the Data Warehouse Terms in the Data Warehouse
• Data Warehouse Translation Values Data Warehouse Translation Values
• Data Warehouse Public Queries LogData Warehouse Public Queries Log
• Data Warehouse Data DictionaryData Warehouse Data Dictionary
• Running a Query Running a Query
• Downloading Query Results to a CSV Text Downloading Query Results to a CSV Text FileFile
• Printing labels using PeopleSoft QueryPrinting labels using PeopleSoft Query
• Creating Pivot Tables in Microsoft ExcelCreating Pivot Tables in Microsoft Excel
Once it’s done, it’s not done
• TrainingTraining
• DocumentationDocumentation
• User EvaluationUser Evaluation
• New queriesNew queries
• SupportSupport
• TweakingTweaking
What Else?
• Added a table for degrees awarded to the Added a table for degrees awarded to the StudentStudent
Data Warehouse in Summer 2004Data Warehouse in Summer 2004
● ● Graduate Admissions Data Warehouse Graduate Admissions Data Warehouse implemented in Summer 2004 implemented in Summer 2004
• Undergraduate Admissions Data Warehouse Undergraduate Admissions Data Warehouse
being implemented in January 2006being implemented in January 2006
Student Demographics
STDNT_CAR_TERM PERSONAL_DATA
PeopleSoft Tables Data Warehouse Table
UADW_STDNT_DEMO
NAMES CITIZENSHIP TERM_TBL
STDNT_ENRL ACAD_PROG
RESIDENCY_OFF CITIZEN_PSSPRT
STDNT_GRPS_HIST VISA_PMT_DATA
VISA_PERMIT_TBL COUNTRY_TBL
STATE_NAMES_TBL UASA_CSC_TBL
XLAT
16 16 TablesTables
Student Addresses
PeopleSoft Tables Data Warehouse Table
STDNT_CAR_TERM ACAD_PROG
UADW_STDNT_ADDR PERSONAL_PHONE PERSONAL_DATA
TERM_TBL ADDRESSES
EMAIL_ADDRESSES
7 Tables7 Tables
Student Academic Info
PeopleSoft Tables Data Warehouse Table
UADW_STDNT_ACAD
STDNT_CAR_TERM TERM_TBL
ADM_APPL_PROG ACAD_STDNT_ACTN
ACAD_PROG ACAD_PLAN
ACAD_SUBPLAN ADM_APPL_DATA
STDNT_ENRL ENRL_REQ_DETAIL ACAD_PROG_TBL
STDNT_ADVR_HIST UASR_AVN_TBL ACAD_PLAN_TBL
XLAT UASR_STDNT_STAT
EXT_ORG_TBL EXT_DEGREE
ACAD_PROG_OWNER ACAD_PLAN_OWNER
NAMES
21 21 TablesTables
Student Attributes
PeopleSoft Tables Data Warehouse Table
UADW_STDNT_ATTR STDNT_CAR_TERM
ACAD_PROG STDNT_ATTR_DTL
TERM_TBL
4 Tables4 Tables
Student Groups
PeopleSoft Tables Data Warehouse Table
UADW_STDNT_GRPSSTDNT_CAR_TERM
ACAD_PROG STDNT_GRPS_HIST
TERM_TBL
4 Tables4 Tables
Student Enrollments
PeopleSoft Tables Data Warehouse Table
STDNT_CAR_TERM ACAD_PROG
UADW_STDNT_ENRL TERM_TBL STDNT_ENRL
ENRL_REQ_DETAIL CLASS_TBL
6 Tables6 Tables
Course Characteristics
PeopleSoft Tables Data Warehouse Table
CLASS_TBL CRSE_OFFER
UADW_COURSE CLASS_INSTR
CRSE_ATTRIBUTES NAMES
PERS_NIDTERM_TBL
CRSE_CATALOG
8 Tables8 Tables
Statistics
UADW_STDNT_DEMOUADW_STDNT_DEMO 50,802 UADW_STDNT_ADDR50,802 UADW_STDNT_ADDR 50,802 UADW_STDNT_ACAD 50,802 UADW_STDNT_ACAD
227,843227,843UADW_STDNT_ENRLUADW_STDNT_ENRL 366,789 366,789 UADW_COURSE UADW_COURSE 39,29239,292
* As of September 2005
40
But what about ??But what about ??
• Official ‘third week’ dataOfficial ‘third week’ data• Financial Aid dataFinancial Aid data• Faculty/Staff dataFaculty/Staff data
QUESTIONS?QUESTIONS?
Marybeth SalmonMarybeth SalmonDirector, University Applications DevelopmentDirector, University Applications DevelopmentUniversity at Albany, State University of New YorkUniversity at Albany, State University of New YorkE-mail: E-mail: [email protected]
Shahnaz SadeghiShahnaz SadeghiTechnical Lead – Data WarehouseTechnical Lead – Data WarehousePeopleSoft AdministratorPeopleSoft AdministratorUniversity at Albany, State University of New YorkUniversity at Albany, State University of New YorkE-mail: E-mail: [email protected]
Wendell LorangWendell LorangDirector of Institutional ResearchDirector of Institutional ResearchUniversity at Albany, State University of New YorkUniversity at Albany, State University of New YorkE-mail: E-mail: [email protected]
ContactsContacts