current state analysis: data and application...

Current State Analysis: Data and Application Interoperation

Final Report

February 2015

Agenda

2

•  Current-State Analysis Objectives •  What We Did •  Directory Data Flows •  Name and Address

•  Directory Information Exchange Formats •  Creating and Transmitting Information •  Technologies for Transforming and Moving

•  SIS Current Production Data Flows •  Representative Financial and HR Data Flows •  Key Observations •  Benefits of Interoperation •  An Approach: The Concept •  Practical, Use-Case Driven Approach •  Next Steps and FY15 Objectives

Current-State Analysis Objectives

3

•  General support for two EA objectives:

•  Provide common approaches for integration across enterprise applications, processes, and data

•  Identify redundant or conflicting processes and data across organizations

•  Specific current-state analysis goals:

•  Current-state analysis of data passed between enterprise applications

•  Options and approaches for integration across applications

Data CollectionPlanning & Coordination: Group Feedback

Analysis Reporting

Objectives

Scope

What to Collect & Why

How to Collect

Key Participants

Schedule

Cross-University Team: ATS (including HDW), IAM, SIS, HBS, HLS, GSD, Vice Provost for Research, Research Computing

▪ Current-State Analysis Wiki

▪ Final Report▪ Enterprise

Architecture Executive Committee Presentation

▪ Stakeholder MeetingsQualitative

▪ Issues impacting our ability to share data

▪ Areas where improvement is desired

▪ Identification of areas of unmet need

Quantitative ▪ Data and formats

exchanged▪ Protocols, vendors

and technologies used; improvement is desired

▪ Consumers of data▪ Sources of data▪ Update/cache

policies▪ APIs available/used▪ Security level of data▪ Authoritativeness

▪ Common formats, protocols and technologies

▪ Technology and system acquisition decision factors

▪ Cost of current state: where/how costs are distributed

▪ Duplication of effort and resources

▪ Resources required/available in departments, schools, and major initiatives

▪ Policies and procedures, particularly change management and decision making

▪ Reported problem areas

▪ Complexity▪ Options and

opportunities not currently available

▪ Our history

What We Did

4

Directory Data Flows

5

Representative Cross-University Directory Data Flows

GMAS (and others - e.g, Library Campus Services, HUHS)

AthleticMembership

SIS +

DCE

IAM

AspireFAS

SPH

PeopleSoft

Oracle Financial& Procurement

HBS

HKS HLS

Multiple HBS Locations

Multiple HBS Sources

AddressPersonName

Email AddressDirectory Listing Data

JobRole Employee Job Code

Role Employee Department Code

PeopleSoft Input XML

DB View

Human Input

Pipe Delimited

Oracle Datapump File

Fixed Length Field Text File

Data Format

Transfer Protocol/Creation MethodOnline forms/PeopleSoft Proprietary

SFTP

Web service

SQL query and SQLLOAD

Employee File

Address DataDegree Data

Bio Data


Bio Data

Person Type

Person Type

Email Address, Directory Listing Data, Name, Person

Address

FAS Phone Name

Name, DOB, last 4 of SSN, UUID, HUID, Gender

CAADS

NamePerson

AddressesDepartment Codes

Email AddressesEmployee Class Codes

Employment StatusHarvard Positions

Job CodesPeople

PeopleSoft Location Codes

Phones

Name TypeName PrefixFirst Name

Middle NameLast Name

Name Suffix15 years service (T/F)

(from secondary DB)

Address, Person, Name, Email Address, Directory Listing Data,Job, Role, Employee Job Code

Department Code

NamePerson

HTTPS Post

AddressPersonName

Email AddressDirectory Listing

DataJob

XML IAM Format

XML HBS Format

Version 1.6 - February 4, 2015


Bio Data

XML PeopleSoft New Hire Schema

School Financial Aid OfficesSchool Financial

Aid OfficesSchool Financial Aid OfficesSchool Financial

Aid Offices

University Financial Aid Liaison Office

School Data

SIS Data

Name, DOB, last 4 of SSN

RESTful FindPerson (JSON) service

FindPerson/JSON

Campus Solutions

Name and Address: Directory Information Exchange Formats

6

XML IAM Format XML HBS FormatXML PeopleSoft New Hire Schema PeopleSoft Input XMLOracle DB View Human InputFile Pipe Delimited Oracle Datapump FileFixed Length Field Text File Find Person/JSON

Name and Address: Creating and Transmitting Information

7

Human Input Forms SFTPSQL query and SQL Load Web ServiceHTTP Post Restful Find Person

Name and Address: Technologies for Transforming & Moving

8

Unix Shell ScriptPerlJavaOracle PL/SQLInformaticaOracle SQL/LOAD & QUERYOracle Datapump UtilityOn-Line FormsOracle & Other Proprietary

SIS Current Production Data Flows

9

SIS Current Production Data Flows

SIS/Campus Solutions

Pipe Delimited


Tab Delimited Admission Data

Data Format

Transfer Protocol/Creation MethodSOAP/XML

SFTP Into SIS

Web REST/JSON FindPerson service

JSON - (FindPerson)

XML SIS Standard Admission XSD

Version 1.3 - January 22, 2015 - Jon Saperia

GSD HKSGSE

New Admit Bio Data

HUID Assignment

ESCI

NameAddressBio Data

Academic Progress

CAADS(Monthly Feed)

Find Person Service

NameAddress

Degree Data (CPP)

PeopleSoft HR to SIS XML

PeopleSoft Payroll to SIS XML

PeopleSoft HRPeopleSoft Payroll

Payroll Data

Bio Data, Workforce, Dept., Job,

Location

New Admit Bio Data

HBS(MBA)

HMS

HBS(Doctoral)

Harvard School of Dental Medicine

SFTP from SIS

HUID Assignment

GSASHarvard CollegeHSPH

HLS(Graduate Programs)

Harvard Divinity School

HLS(JD)

HUID Assignment

New Admit Bio Data

Name, DOB, last 4 of SSN

Name, DOB, last 4 of SSN, UUID, HUID, Gender

Representative Financial and HR Data Flows

10

Representative HR and Financial Data Flows

Oracle Financials & Procurement

Pipe Delimited Text


Tab-Delimited Gift Data

Data Format

Transfer Protocol/Creation Method

SFTP

Version 1.3 - February 6, 2015 - Jon Saperia

HDW

GMAS

PeopleSoftHCM

Campus Solutions

GMAS Other Departments

AP Invoice Feed

AP Invoice Feed

GMAS SAA

Direct Deposit

Data

IDMS Data

Schools

Gl Extract File

Gl Extract File

PersonJob

PaycheckDirect Deposit

Online forms/PeopleSoft Proprietary

Human Input

VoucherDirect

Deposit

Eureka PersonTraining

CSV Text File

University Financial Aid Liaison Office

HTTPS Post

VoucherDirect

DepositHUID

XML TOOR Feed

Tubs, Orgs, Objects, Roots

XML Feed to GL

Funds, Activity, Sub-ActivityGift Data

Advance Application

Direct DB Feed

Direct Oracle Data

AR Invoice Feed

AR Invoice Feed

Direct Oracle Data via Informatica

Chart of Account

Data

Gift Data

Key Observations

11

•  “Stovepipes” of information and data flows were created as a result of local optimizations, resulting in complexity

•  Local optimizations are easier locally; globally, they’re more expensive

•  Interoperation guidance must inform how we acquire and develop our systems and technology

•  Technology is only part of an effective solution — enterprise architecture, software/systems engineering, governance, and operations must all be included for effective progress

•  Interoperation is not currently seen as a service, but creating and operating such a service to serve the entire University is preferable

•  We must work together to develop efficient and effective communication with all teams and interested parties across the University

Benefits of Interoperation

12

Stakeholder Experience Today Imagine …

Service Users: Faculty, Students, and

Staff

•  Time-consuming, repetitive entry of biographical and other data for each application/service used

•  Inability to answer simple business questions (i.e. “how many of X does the University have?”)

•  Difficult and time-consuming to access information not already provided via an existing flow to a student’s, faculty member’s, or staff’s organization

•  Users do not have to re-enter the same data — applications access a real-time service to get necessary information

•  Faculty, students, and staff can ask questions on an ad-hoc basis and get reliable answers

•  Faculty, students, and staff have flexible, timely access to data across the University

IT Systems and Policy Decision-makers

•  No guidance about interoperation requirements; all decision-making about technology acquisition or development is local, which increases OP/CAPX

•  No facility for cross-University data (i.e. “what do we mean by ‘address’ or ‘student’?”)

•  Decision-makers have a simple checklist of interoperation requirements to help them make local decisions informed by University-wide objectives

•  We have a common understanding of what our data means across the University

Systems and Software Engineering

•  Each integration is custom-coded, which increases technical risk

•  So much time is spent on basic integration that tasks with greater user-perceived benefit are delayed

•  Schools and departments have a set of reusable integration tools

•  More efficient access to information — and time freed to access it — free up time to develop more user-valued services

Operations

•  Multiple ad-hoc streams create complexity for each system — time-consuming and brittle

•  Difficult to identify faults and restore service due to many file transfer servers/flows with hard-to-understand interdependencies

•  A centralized interoperation service is available across the University, operated by a focused team

•  No longer necessary to deploy, configure, and operate point-to-point integration services

Cost-efficient, flexible access to consistent information across the University that is easy for users and less complex for engineering and operational staff.

An Approach: The Concept

13

•  Effective interoperation requires collaborative actions across many organizations

•  Each area can be addressed individually; must be coordinated for effective progress

Service and Data Integration

Operations (5)

Operations

Data Warehouse

Information Bus

Other Resources

Operational Engineering

Training

Outreach Support - Engineers on Loan

Enterprise Architecture (1)

Technologies for Integration

Architectural Principles

Design Patterns, etc.

API’sDocumentation

Change Management

Standards

Educational (e.g., eduPerson)

Other de facto/de jure Standards

System/Technology Acquisition

Selection and Acquisition Guidelines

Exception Processes

Organizational Structure and Approaches (4)

Cross-Team Coordination

Governance & Analysis (3)

GovernanceMaster Data Management

Data Governance

Data Analyses/Tools

Ad hoc analysis

Routine Analyses

Metrics/Time Variance/Series and Cross Subject Correlation

Snapshot Operational Detail/Audit

Software and Systems Engineering (2)

Data Warehouse

Information Bus

Data Marts

Data Maintenance and Management

Example Code/Libraries

Automated Instrumentation

Modular Approach to Data/Service Integration

Jon Saperia - 1/13/2015 - Version 1.3

Practical, Use-Case-Driven Approach

14

Ente

rpris

e Ar

chite

ctur

eEn

gine

erin

gG

over

nanc

eO

pera

tions

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4CY15 CY16 CY17

Use Case Implementation Approach to Implementation of an Integrated Interoperation Environment*

Chart of Accounts Access Biographical Data 1 HR Data 1 Biographical Data 2 HR Data 2 Accounting Data 1Use Cases

Jon Saperia - Version 1.0 - January 27, 2015 * Use cases are examples based on current state analysis problem areas, additional input added over time.

System/Technology Acquisition & Exception v1.0 System/Technology Acquisition & Exception v2.0

Work done across use cases. Use case specific work.

Requirements v1.0 Requirements v2.0

APIs, Principles, Patterns, and Standards

APIs, Principles, Patterns, and Standards

APIs, Patterns, and Standards




Eng. Infrastructure (warehouse, marts, bus)






Release-specific libraries, APIs, tools & Instrumentation






Governance Framework v1.0 Governance Framework v2.0

Master Data Management policy





Data governance Data governance Data governance Data governance Data governance Data governance


Deployment and operations environment v1.0 Deployment and operations environment v2.0

Outreach and Training

Release-specific deployment and operations











Planning & Coordination

Next Steps: FY15 Objectives

15

•  Creating an integrated plan

•  Building initial team and communications

•  Engineering activities:

–  Identify core team members and their roles

–  First pass at detailed requirements

–  Evaluate our technical options and abilities

–  Refine the plan

Thank you!

current state analysis: data and application...

Documents