chart your course: develop adaptive analytics technology ... › files › conferences_and_training...

Chart Your Course: Develop Adaptive Analytics Technology Responsiveness

NSAA 2020 Annual Conference June 3-4 Virtual Training State Auditor’s Office of Massachusetts

Co-Presenters:Tom Meier, CIO

Penny Dillon, Director of Data AnalyticsThursday, June 4, from 4:10 – 5:00 p.m. Eastern Time

Making Government Work Better 23/21/2017

Audit Analytics Program“The price of doing the same old thing is far higher than the price of change.” Bill Clinton

“The (new structure) will allow us to create a faster and stronger multidisciplinary team, which can drive innovation and innovative culture quicker and faster.” Alastair Curtis

“Fostering a culture based on creativity and collaboration helps bring down the walls that divide people from each other within organizations. Collaboration is not just amongst and within a given team; it needs to filter through and cross teams and departments. Creativity is not relegated to the creative department; it is a requirement for all.” Cynthia Thomas

Making Government Work Better 33/21/2017

Audit Analytics Program

AUDIT

ANALYTICSDATA

Making Government Work Better 43/21/2017

Audit Analytics Program

AUDIT

ANALYTICSDATA

Making Government Work Better 53/21/2017

Impact on Your Audit Analytics Program

Strong technology foundation

Data and analytics maturity

Depth and breadth of analytic capabilities

New roles and responsibilities

Digital strategy

Making Government Work Better 63/21/2017

Enable Quick Solutions

Ad hoc small scale solutions

Learning opportunities

Multidisciplinary cohort

Culture of experimentation

Fit for purpose

Making Government Work Better 73/21/2017

Alignment with Audit Process is Key

• Planning/Audit Analytic Plan • Data sharing/Acquisition• Data Cleansing/Preparation/Normalization• Preliminary Analysis

Planning phase

• Support Procedures• Execute Analytical Procedures• Ad Hoc Analysis

Fieldwork phase

• Report Visualizations

Reporting phase

Making Government Work Better 83/21/2017

Making Government Work Better 93/21/2017

Response to Innovation

Handle the Hype

Strategic capability Contextualize for your critical business scenarios and

outcomes Low hanging fruit/Use cases that

are not too complex (simple process, structured data)

Pilot/POC with vendor support

Scale up/mature over time

Making Government Work Better 113/21/2017

C & CCohortsChange

Summary

Audit

AnalyticsData

Supports

Alignment Innovation

THO

MA

S M

EIER • CIO and Director,

Massachusetts State Auditor’s Office

• Executive Manager for IT and Information Security for over 12 years.

• Network Architecture with IoT research and product development for 4 years.

• Member, author and editor for the Hydraulic Institute.

• IEEE, ISACA, Infragard and ACM

Thomas.Meier@sao.state.ma.us

Understanding the Audit Mission

Understanding the Audited Agencies

Creating the Digital Infrastructure

Selecting the right goals Chart The Course Executing the plan

Making Government Work better

Government agencies in digital transition

Know the lay of the Technical land

Technology, Process, Policy Hearts / Minds / Budget Listen, Learn, Lead

Digital Landscape and Adaptation

Tactical Strategic

Adapting to Information Digitization Was about paper and scanning

> Now it’s about OCR and Meta Data Was about access to audited agency applications and

databases > Now direct connections to database tables for

analysis and complex analysis tools Was about referrals and mandated audits

> Soon to be about AI motivated prescriptive auditing

Adapting to Make Better Connections Audit Agency Data Models

› On Premises› In the Cloud› Secured Data › Paper or other Documentation

Adaptive Future Models and Processes› AI Systems› Cloud to Cloud Systems› Predictive Referrals

Data Analytics and Information Technology

Analytics teams focus on identifying the data, cleaning it and creating relationships that Auditors can use

IT creates the environment and pathways for the analytics teams to operate and deliver their value added or process methods to Auditors. IT must have resources and frameworks to support those needs.

For Information Technology to Add Value

Processes, Policies and Skills to setup and operate the enterprise systems quickly, securely and efficiently

Key Skills are › Cyber Security› Data Base Engineering› Network Management› Project Management› Software Engineering› Business Analysis› Cloud Innovation› Training and Support

Traits for the IT Technologist

People› Managers that understand the Audit Model› CISM’s for security› Technology Leads to execute

plans› Innovation and Solutions

Architects› Business Analysts› Data Analysts

Traits for the Technologist

Curiosity and a willingness to Learn Sense of Purpose, uphold standards

and a willingness to fulfill duties Willingness to take on Risk Organization in solving problems, see

issues from other’s perspective Communication skills without jargon Grounded and Realistic Optimism

Sow Your Own Talent Look in your back yard – many resources will be

undiscovered Talk to your staff and see where they have aptitude Skill-up trainable staff Re-organize those you cannot Set Aggressive but Realistic Goals Create focused policies and procedures Make your staff responsible and give them some

latitude

Attendance Check

Audit, Analytics and Technology

Data Analytics spends most of its time finding and collecting data. Now that they found it – what?› Analysis› Algorithm Development› Dialog with Data Creators, Auditor, Visualization Experts, IT

Enterprise IT spends most of its time creating and running an agile Infrastructure of Premises and Cloud resources. They create connections with people, technology and services in support for Auditing and Analytics. Service Orientation!

Audit, Analytics and Technology

What tech is needed to support the Audit› Journey Map the Analysis, Data and Results that support Audit

Goals› Audit Process, Analytics Process, IT tools and achieved goal

Agency Data

Other DataPublic Data

Documents

Various Data Sources

Focus DataAcquisition

ToolsSelection

Skilled GoalAchievement

Technology Impact› Network and Data Security› Data integrity and Governance› Network Bandwidth› Process and Policy› Cloud and Premises Technology

Architecture› Archiving and Public Information

Requests

In Support of the Audit Model

Hybrid Infrastructure on Premise

• Lots of cores• Lots of Stores• Fast Network• Limited Costs• Watchful Systems

On Premises Computing

Most Adaptive Infrastructure

• Looks like Ours – Looks Like Theirs

• Cloud To Cloud Connect

• Compute in the Cloud Return Summaries

Internet

Generalized OSA Network

ExternalData

AuditedAgencyData

The Cloud Portion

OSASystems

Agency &ExternalSystems

Power AppsQlikTableau

Adaptive Platform Strategy

Right size your Platform Technology› Cloud from Microsoft, Google or AWS› Linux and Windows› Develop in Python or C#› Low code – No Code› Mobile and Server

Know what you are getting into with proprietary platforms that don’t get along.

Adaptive Tool Selection

Use different tools when you have to› ACL vs. Teammate Analytics› Qlik vs. Power BI vs. Tableau

Best tools for analysts that are no-code or low-code Audit Report Scheduling and Creation Systems Analytics systems and templates for Auditors Training and support on line and in person Communication Platforms for Collaborating

Filter out the Hype Right size your technology solutions for the size and

complexity of your audit process. Resist the temptation to add technologies or solutions

for problems that don’t exist yet. Use common sense and all technologies to solve your

problems. Paper and pencil are also technologies –they are still good ones.

Synergy with agencies that are being audited is key. Keep a good relationship and regular contacts with key individuals.

Filter out the Hype

Communications are key to advancing and implementing good technologies.

Document and support good algorithm development with your Analytics Teams› Promote Story-Telling: Findings to lay groundwork, present

findings, link findings to data and sources and substantiate next moves.

In Conclusion

Supporting your Data Analytics teams with the right Enterprise IT summons the bright digital future that will make government better.

Making Government Work Better 333/21/2017

Thomas Meier, CIOthomas.meier@sao.state.ma.usTEL: 857-242-5619

Thank you! Our Contact Information

Penny Dillon, Director of Data Analyticspenny.dillon@sao.state.ma.usTEL: 857-242-5519

Office of the State Auditor One Ashburton Place, Room 1819Boston, MA 02108

www.mass.gov/auditorTwitter: @MassAuditorfacebook.com/MassAuditor

Making Government Work Better 353/21/2017

Reference Images https://www.secsign.com/wp-content/uploads/2016/12/SecSign_onpremise_cloud.jpg http://1.bp.blogspot.com/-V5668kM1TDw/UUqWcVSCcXI/AAAAAAAAAMc/mK_B4RExZbE/s1600/Thoughtful-Woman.jpg https://vividlife.me/ultimate/wp-content/uploads/2013/05/Untitled85.jpg https://hiring.monster.com/employer-resources/recruiting-strategies/talent-acquisition/how-to-source-interview-it-managers/ https://www.fastmetrics.com/blog/tech/what-is-cloud-computing/https://docs.microsoft.com/en-us/azure/architecture/reference-

architectures/data/enterprise-bi-adf https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/data/enterprise-bi-adf https://image.shutterstock.com/image-photo/happy-little-girl-big-sunglasses-260nw-1035651334.jpg http://contentz.mkt5790.com/lp/2842/268211/Foundation%20IAReponseToDisruptiveInnovations%20June%202019_0.pdf Gartner.com https://i2.wp.com/freemasonrymatters.co.uk/wp-content/uploads/2015/07/QA-Graphic.png?ssl=1tner.com