City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

City Services Auditor (CSA)

Highlights of FY 2017-18 Audits & Projects

Presentation to CGOBOC January 22, 2018

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Work Planning Process

• CSA’s work plan is driven by: Charter and Administrative Code requirements; limited risk analysis; Mayor, Board, leadership, and department requests; and bond and capital programs

• CSA prioritizes projects and audits in consultation with the Controller, leadership, and departments, and considers risk, while balancing against available resources

TIMELINE FOR CSA WORK PLANNING FOR FY 2018-19

2

Task Month• Conduct department work planning and staff brainstorming meetings March–April• Develop and compile proposed projects May• Complete FY18-19 Master Work Plan June• Conduct resource balancing Mid-June• Confirm projects with departments End of June

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Issued Projects (July – Dec 2017)

3

Issued Date Report

12/7/2017 Performance Scorecards Update and FY2016-17 Performance Measures

12/5/2017 Park Maintenance Standards Annual Report (FY2016-17)

11/14/2017 FY2016-17 Nonprofit Monitoring Program Annual Report Highlights

11/14/2017 FY2016-17 Nonprofit Monitoring Program Annual Report

11/13/2017 FY2016-17 Nonprofit Monitoring Results Dataset

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Issued Projects (July – Dec 2017)

4

Issued Date Report

10/12/2017 Comparative Analysis of Modular Housing Options for San Francisco’s Homeless

8/30/2017 Adult Probation Department Reentry Division CASC Program Analysis

8/24/2017 Management & Employment Practices Program Update

8/15/2017 Annual General Obligation Bond Program Report, Fiscal Year 2016-17

8/3/2017 Results of 9-1-1 Dispatcher Training Analysis

7/11/2017 SFMTA Public Participation Plan - Project Overview

7/11/2017 SFMTA Language Assistance Plan - Project Overview

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Muni Customer Service

5

Results and Findings

The cross-divisional project team met monthly over four months to improve the process by:• Creating standard guidelines for processing key types of feedback• Creating a protocol for SF311 intake employees to ensure all information necessary to

resolve a complaint is collected from the customer at intake• Creating templates so that MCS employees can respond quickly to the public about the most

frequent types of PSRs.• Developing criteria for priority video pulls that are critical to investigations.• Selecting metrics for a new weekly reporting dashboard to monitor performance and manage

daily operations.• Building a SharePoint Knowledge Base to serve as a central repository for the work products

from the project, along with other relevant materials for efficient processing of complaints.

Why We Did the ProjectAt the request of SFMTA, CSA convened staff from the Transit and Muni Customer Service (MCS) divisions to improve the process for responding to customer feedback about Muni. The project focused specifically on employee-related feedback (ie drivers), which comprises approximately 60% of all feedback submitted to Muni.

Staff from SFMTA Transit, MCS, and Performance & Business work to identify obstacles and develop

solutions to better serve the pubic.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Modular Housing

6

Results and FindingsModular construction offers potential modest cost savings and significant time savings, however:• Trade leadership is concerned about the quality of modular units manufactured outside of

San Francisco and does not want union members to lose job opportunities.• Modular manufacturing is currently a volatile and unstable industry.• It is risky to use modular technology developed by inexperienced companies.

The City and trade leadership should continue to discuss the idea of using modular methods in San Francisco for a limited number of short-term projects.

Why We Did the ProjectThe Mayor’s Office requested that City Performance study potential cost and time savings associated with using modular methods to build housing for the homeless in San Francisco.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Nonprofit Monitoring and Capacity Building Program

7

Results and Findings• More contractors conformed with standards: In FY16-17, 84% of contractors were in full

conformance with monitoring standards, leaving just 21 contractors with one or more findings. This represents a significant improvement over prior years (typically 71%).

• Fewer contractors with significant findings: In FY15-16, 16 contractors had 5 or more findings at the close of the cycle, while in FY16-17, just 6 contractors had 5 or more findings. Many contractors with a high number of findings received technical assistance to support them toward conformance with City standards.

• Technical assistance leads to fewer findings: CSA funded 250 hours of individualized coaching for 17 nonprofits in FY16-17. On average, nonprofits that received coaching had 8.1 fewer fiscal findings in FY16-17 compared to FY15-16. The average for all nonprofits was 2.5 fewer fiscal findings year over year.

Why We Did the ProjectCSA coordinates a Citywide program to jointly monitor the fiscal health of nonprofit contractors and provide technical assistance when nonprofits do not meet financial management standards. The program monitors about 150 nonprofits annually, representing $525M in funding from 10 departments.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Scorecards and Performance Measures

8

Results and FindingsRecent Scorecards highlights:• Response to 911 calls has improved since April 2017, averaging 81% in FY 2017-18, up from

74% in FY 2016-17, even as call volume continues to increase. The Department of Emergency Management is implementing several strategies to improve response, including a major overhaul to hiring and training practices as well as redirecting some calls to SF311.

• Traffic fatalities were down in 2017 compared to the prior year – 20 in 2017 compared to 30 in 2016 – the lowest level since 2009. Vision Zero SF, a multidepartment initiative, aims to eliminate all traffic deaths by 2024 through engineering, enforcement, and education efforts.

Why We Did the ProjectCSA continues to maintain the San Francisco Performance Scorecards website, updating measures monthly across eight policy areas. Starting this month, the Performance Program team is working with all City departments to align centrally-tracked performance measures with the Mayor’s Office’s Strategic Initiatives Plan Framework in time to be included in the fiscal years (FY) 2018-19 and 2019-20 Mayor’s Budget Book.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Performance: Current Major Projects

9

• Interdepartmental Working Group and Incident Command for Street Homelessness & Response

• Police Patrol Staffing Analysis

• New City Office Building – Permit Center Planning

• Public Health – Metrics, Managed Care, Electronic Health Records

• LEAN Program; Training, Rec/Park Maintenance Yard Partnership, Rapid Improvement Events

• Library Open Hours Assessment

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

Highlights of CSA Audits’ Ongoing Audit Work

10

Performance Citywide Compliance

• Sheriff IT Governance Audit• Library IT Governance Audit• DPH Environmental Health Audit• Citywide Fleet Management

Audit• Citywide Facilities Audit• 311 Call Center Audit• Citywide IT Procurement Audit• SFMTA Workers Comp Audit • Divisional and Operational

Audits, Assessments, & Reviews

• Cash Collection Audits• Cash Disbursement Audits• Payroll Audits• Contract Compliance Audits• Inventory Audits• Nonprofit Provider Audits• Eligibility Compliance Audits• P-Card Monitoring Program• Concessions & Mandated Audits • Financial Audits and Reviews

Construction

• 2008 and 2012 Parks Bond Expenditures Audits

• 2014 ESER Preconstruction Phase Audit

• SFPUC Sewer System Improvement Program Preconstruction Phase Audit

• DPW and Port Construction Contract Close-out Audits

• Bond Oversight Benchmarking• Construction Insurance Study

IT & Systems Audits• Ongoing Systems Security

Audits and IT Controls Audits

Whistleblower Investigations• Ongoing Whistleblower Program

Investigations and Activities

Other Assessments• Recommendations Follow-up• SFPD DOJ Audit Implementation

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Audits: Issued Audits (Jul – Dec 2017)

11

Issued Date Report

12/28/2017 Ethics Commission: Nine Candidate Committees Substantially Complied With and Three Candidate Committees Did Not Comply With State and City Campaign Finance Laws During the November 8, 2016, General Election

12/21/2017 Office of the Assessor-Recorder: Audit of the Social Security Number Truncation Program11/9/2017 SFMTA: Cypress Security LLC Could Not Demonstrate Compliance With Contract Requirements Including

Training, Health Benefits, and Minimum Compensation for Its Subcontractors’ Guards11/8/2017 City Services Auditor Follow-up on Recommendation Implementation Status - Fiscal Year 2017-18, First Quarter

10/24/2017 SFMTA: The Information Technology Division Is Successfully Becoming a Strategic Partner but Needs to Improve Governance and Staffing

10/23/2017 Citywide Cash Transactions: Combined Report of 15 Cash Collection Locations of 5 City Departments Audited in Fiscal Year 2016-17 — Weaknesses Exist in Some Cash-Handling Processes

10/19/2017 Results of Construction Reports in Fiscal Years 2014-15 Through 2016-17

10/11/2017 Port Commission: Mama Franceschi, L.P., dba Capurro’s, Underpaid $682 in Rent to the Port for 2011 Through 2013

10/11/2017 Port Commission: Pier 23 Café, Inc., Underpaid $16,987 in Rent to the Port for 2011 Through 2013

9/26/2017 SFMTA: IMCO Parking LLC Correctly Reported Fifth & Mission/Yerba Buena Parking Garage Revenues for July 2014 Through June 2016, but a Few Improvements Can Strengthen Its Operations

9/19/2017 Airport Commission: Guava & Java Inc. Reported Its Revenues and Paid Rent for 2014 and 2015 per Its Lease but Did Not Retain Some Sales Records and Submitted a Required Annual Report Late

9/14/2017 City Services Auditor Recommendations Not Implemented After More Than Two Years on June 30, 20179/12/2017 Bulletin: Results of Eligibility Screening Audit Program

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA Audits: Issued Audits (Jul – Dec 2017)

12

Issued Date Report

8/30/2017 City Services Auditor Follow-up on Recommendation Implementation Status Annual Summary – Fiscal Year 2016-17

8/28/2017 Department of Homelessness and Supportive Housing: Increased Oversight, Fiscal Sponsorship Controls, and Accountability Are Needed to Improve United Council of Human Services’ Operations

8/10/2017 San Francisco Housing Authority Should Reorganize Its Finance Department to Better Meet Its Organizational Needs and Improve Its Financial Management Controls, Systems, and Staffing Strategies

8/8/2017 Airport Commission: SkyWest Airlines, Inc., Paid Its Landing Fees in Accordance With Its Operating Permit for July 1, 2013, Through June 30, 2015

8/3/2017 Larkin Street Youth Services Adequately Identifies Program Expenses but Needs to Strengthen Its Internal Controls

7/31/2017 Cash Fare Collection Procedures and Controls on SFMTA Cable Cars Are Inadequate, Creating Opportunity for Fraud and Theft

7/28/2017 External Quality Control Review of the CSA Audits Unit for 2014 Through 2016

7/27/2017 Human Services Agency: Edgewood Center for Children and Families’ Residential Treatment Program Cost Proposal Is Adequately Supported by Its Financial Records

7/26/2017 Office of the Treasurer and Tax Collector: Quarterly Review of the Schedule of Cash, Investments, and Accrued Interest Receivable as of March 31, 2017

7/10/2017 Port Commission: Sitting By, Inc., dba Hi Dive, Underpaid $5,459 in Rent to the Port for 2011 Through 2013

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA AuditsSFMTA Cable Car Cash Fare Audit

13

What We FoundSFMTA’s cable car fare collection practices are unsatisfactory due to inadequate controlsand unmanaged risks. This is a significant problem because more than $10 million of cable car fares in fiscal year 2015-16 were collected in cash. CSA found:• Two cable car conductors failed to give receipts for cable car fare payments, which

increases the potential for theft. One of these conductors was ultimately arrested on suspicion of felony misappropriation of public money and embezzlement. Another conductor not identified in the audit was also arrested on similar charges.

• Conductors failed to collect fares from CSA auditors on 11 (37 percent) of the 30 sample cable car rides.

• Conductors did not collect from an estimated 178 other passengers during the 30 rides, resulting in forgone revenue of as much as $1,323.

Why We Conducted the AuditAt SFMTA’s request, CSA audited cable car fare collection practices as part of its 2016-17 cash-handling audit program. CSA also conducted this audit in 2007 and a follow-up review of that audit in 2010.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA AuditsResults of Construction Reports

FY14-15 Through FY16-17

14

Key ResultsCSA conducted 15 construction-related audits, assessments, and reviews of six city departments during July 2014 through June 2017. Below is a summary of all CSA report findings and identified risks related to construction projects.

Reported Finding & Mitigation Risk to Be Managed and Minimized

Some departments should strengthen change order documentation and adhere to contract requirements.

• Departments risk incurring a liability for additional, unneeded work, increased costs, and delayed payments to the contractor due to financial disputes.

• It may be difficult for the department or a project manager to manage costs and variances.

• Departments may have less legal recourse against contractors' bonding companies if problems with work arise.

Some departments do not have a proactive, strategic approach to construction safety management and do not always consider safety as a key component in the construction bidding and contracting process.

• Departments may award contracts to bidders that are irresponsible in terms of safety, leading to safety incidents and injury lawsuits.

• Inconsistent communication of expectations across projects could occur. • Contractors may be less likely to experience direct consequences for

allowing unsafe conditions, whether caused by willful violations or carelessness.

• There could be improperly trained staff at construction sites.

Some departments should better monitor costs.

• Project costs could be over- or understated. • Departments may be paying more for work than they should be or

paying for work that is out of scope or unallowable.• Cost overruns may lead to project disruptions or reductions in the scope

of work .

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA AuditsSFMTA IT Governance Audit

15

Why We Conducted the AuditSFMTA restructured its information technology (IT) division to more effectively manage the evolving technological needs of the department. SFMTA requested this performance audit as part of efforts to ensure the successful transition of the division from a help desk function to a strategic partner within the organization. The audit focused on IT governance and staffing and did not include an IT security assessment.

What We FoundSFMTA’s IT division is making itself a strategic partner in department operations but needs to improve governance. A robust IT governance framework consists of the following five areas, of which SFMTA’s IT division meets or partially meets leading practices in only three:

The IT division does not have a dedicated, full-time information security officer, has not adopted a cybersecurity framework, and may need more staff, as indicated by the amount of overtime and backlog of work in the division.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

CSA AuditsPassed Required External Peer Review

16

In July 2017 an external peer review team from the Association of Local Government Auditors assessed the quality controls of the CSA Audits Unit. The team found that the CSA Audits Unit’s internal quality control system is suitably designed and operating effectively to provide reasonable assurance of compliance with generally accepted government auditing standards (GAGAS) for audits during 2014 through 2016.

Passing a triennial peer review with no findings is a testament to CSA Audits’ robust quality assurance program, as well as its audit staff’s overall professionalism, competence, and diligence to ensure GAGAS compliance.

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

Report on Audit Recommendations Follow-Up

• CSA follows up on all recommendations every six months after issuance, consistent with SF Charter requirement, Section F1.105.

• The value in CSA’s work is not only in the recommendations themselves, but in the corrective actions implemented by city departments to address those recommendations, as well.

• Departments implemented 91 percent of the 2,097 recommendations CSA issued from July 1, 2010, through June 30, 2017, and 98 percent of the 1,647 recommendations issued from July 1, 2010, through June 30, 2015.

17

City

and

Cou

nty

of S

an F

ranc

isco

Offi

ce o

f the

Con

trol

ler

QUESTIONS & DISCUSSION

QUESTIONS OR COMMENTS?

18