oregon department of administrative services enterprise goods & … · 2020. 9. 10. · 1...

1 Notice of Intent to Award in RFP DASPS-2232-19 (OED UI MOD Services)

Oregon

Kate Brown, Governor

September 10, 2020 To: Proposers to State of Oregon, Department of Administrative Services, Procurement

Services, Request for Proposals DASPS-2232-19: Oregon Employment Department Unemployment Insurance Modernization Services.

From: Nicholas Betsacon, IT Procurement Strategist and Single Point of Contact Department of Administrative Services, Procurement Services Re: Notice of Intent to Award: Request for Proposals DASPS-2232-19 (the "RFP")

Pursuant to provisions under Section 5.1 of the RFP, Award Notification Process, the State of Oregon, Department of Administrative Services ("DAS"), Procurement Services (“DAS PS”) and the Oregon Employment Department (“OED”) Unemployment Insurance Modernization (“UI MOD”) Evaluation Committee ("Committee") have completed evaluation of all required submissions from the five (5) Proposers who responded to the RFP and its twelve (12) Addenda. Based upon the results of this evaluation, DASPS now announces its intent to award a contract for OED UI MOD Services to FAST Enterprises. This notice of intent to award ("Notice") is conditional.

The State of Oregon will not execute a contract for the prescribed services to FAST Enterprises, unless these parties negotiate mutually acceptable final contract terms and conditions, including scope and price, according to the relevant provisions of the RFP and its Addenda. The parties must satisfy the condition within a reasonable period from the date of this Notice.1

In the event the planned negotiations fail, DAS PS and OED reserve the right to withdraw the Notice and re-solicit for UI MOD Services. In the alternative, DAS PS and OED may choose to commence final contract negotiations, according to the relevant provisions of the RFP and its Addenda, with the next ranking Proposer.

The Committee reviewed, evaluated and scored the Proposers in accordance with a two-phased, multiple-round process. Phase 1 included two rounds as follows: Round 1 for Administrative Proposal Evaluation and Round 2 for Technical Proposal Evaluation. Phase 2 included two rounds as follows: Round 3 for proposed UI MOD Solution demonstrations and Round 4 for 1 DAS and OED consider 45 days from the date of the Notice as a reasonable period of time in which the parties can negotiate and execute an initial contract. DAS and OED reserve the right to extend the established negotiation period, or to terminate it early, depending upon the course of negotiations at any point in time.

Department of Administrative Services Enterprise Goods & Services

Procurement Services 1225 Ferry Street SE

Salem, Oregon 97301 (503) 378-4642

E-Mail: [email protected]


diverse verification and validation evaluations of the two, top-ranking Proposers after Round 3.2 In accordance with RFP provisions, DAS PS and OED determined not to carry over Phase 1 scores to Phase 2; and determined not to carry over Round 3 scores to Round 4.

The following five organizations submitted Responsive Proposals for scored evaluation:

1) Deloitte Consulting;

2) FAST Enterprises;

3) Geographic Solutions;

4) Sagitec; and

5) Tata Consultancy Services (“TCS”).

In Round 1 the Committee members individually reviewed each Proposer’s Administrative Proposal submissions in the context of scoring criteria set forth in the RFP. Thereafter, members submitted their individual scores to the procurement’s Single Point of Contact (“SPC”). The SPC compiled and averaged the individual scores submitted for each Proposer. The average scores became the Proposers’ Round 1 final scores. An excerpt from RFP Table 4.11.4, depicting the Round 1 Administrative Proposal scored components, is set out below:

Table 4.11.4

Round 1: ADMINISTRATIVE PROPOSAL focus 1,500

1 Cover Letter/Executive Summary 50

2 UI Modernization Core Requirements Checklist (Attach I‐1) 450

3 Proposer Experience 400

4 Proposed Leadership and Staff 400

5 Personnel Management and Staffing Plan 200

The Round 1 Scoring Summary, which depicts total scores for each Proposer by Scorer and the Proposers’ respective final scores, is set on the next page.

2 RFP Addenda numbers 7 to 9 set out Round 3 requirements and RFP Addenda numbers 11 and 12 set out Round 4 requirements.


1 2 3 4 5 6 7 8 9 AVERAGE

DELOITTE 1213 1322 1393 1495 1395 1375 1405 1480 1434 1390

SAGITEC 1221 1198 1249 1387 1361 1291 1298 1363 1384 1306

FAST 1140 1305 1191 1173 1365 1377 1323 1399 1415 1299

GEO SOL 1125 1135 1157 1383 1391 1273 1315 1373 1371 1280

TCS 1114 1212 1126 1173 967 1249 1182 1234 1008 1141

DAS PS and OED determined to move all five Responsive Proposers to Round 2 Evaluation, Technical Proposals. An excerpt from RFP Table 4.11.4, depicting the Round 2 Technical Proposal scored components, is set out below:

Table 4.11.4

Round 2: TECHNICAL PROPOSAL focus 10,250 Points

1 UI Modernization Core Requirements Responses (Attach I‐2) 5,000

2&3 Scope of Services Response (Attach A‐2) and Examples of Work 5,250

i Project Management 500

ii Design Documents, and Services 500

iii Software Components, Development, Configuration, and Integration 500

iv Data Conversion and Migration 500

v Integration/Interfaces 500

vi Organizational Change Management 250

vii Testing 500

viii Usability, Training, and User Support 500

ix Implementation Plan 250

x Maintenance, Support, and Service Levels 500


xi Hosting 250

xii Security: Confidentiality, Availability, and Integrity 500

In Round 2 the scoring members of the Committee (“Scorers”) individually reviewed the entirety of each Proposer’s Technical Proposal submissions in the context of the applicable scoring criteria. In addition, several technical and business subject matter experts on the Committee (“Advisors”) received common, selected assignments from each Proposer’s Technical Proposal submissions for the purpose of reviewing and commenting on observed strengths or areas of possible concern, if any, regarding such assigned submissions. Scorers had access to any Advisor comments for their discretionary reference before completing their Round 2 scoring of Technical Proposal submissions.

Scorers submitted their individual scores to the procurement’s SPC. The SPC compiled and averaged the individual scores submitted for each Proposer. The average scores became the Proposers’ Round 2 final scores, as follows:

1 2 3 4 5 6 7 8 ROUND 2 ROUND 1 ROUNDS 1 & 2

DELOITTE 6650 8380 8345 7524 8038 8746 8339 7795 7977 1390 9367

COMPETITIVE

RANGE

FAST 6445 8409 7494 6819 8512 8590 8309 7496 7759 1299 9058

SAGITEC 7584 8214 7294 7162 7889 8108 7762 6589 7575 1306 8881

GEO SOL 7257 7939 7030 7005 7446 8240 8007 6953 7485 1280 8765

TCS 5271 7516 7056 5263 5184 6593 7285 4850 6127 1141 7268

Round 2 scoring results supported the exclusion of one of the five Proposers from the Competitive Range moving on to Phase 2 of the procurement. Under Section 4.13.2 of the RFP, the excluded Proposer had a protest opportunity, but declined to exercise such opportunity.

The four Competitive Range Proposers moved on to the second and final procurement phase comprised of Rounds 3 and 4. In Round 3, the four Proposers demonstrated their respective UI MOD Solutions in the context of the requirements set out in Attachment 1 to this Notice, Round 3 Demonstration Requirements. Committee Scorers submitted their individual scores to the procurement’s SPC by February 25, 2020. Thereafter, the SPC compiled and averaged the individual scores submitted for each Proposer according to applicable RFP requirements. The average scores became the Proposers’ Round 3 final scores.

The Round 3 Scoring Summary, which depicts total scores for each Proposer by average percentages and the Proposers’ resulting final scores, is set out on the next page.


DEMONSTRATIONS MAXIMUM POINTS Insert Rating % of total points

per scale

Calculated Points - rating %

multiplied by rating points

Insert Rating % of total points

per scale




per scale




per scale



1500 81% 1215.0 83.0% 1245.0 69.0% 1035.0 68.4% 1026.0

500 88.5% 442.5 89.5% 447.5 76.7% 383.5 70.9% 354.5

1500 86.9% 1303.5 91.4% 1371.0 60.4% 906.0 66.3% 994.5

1000 80.9% 809.0 88.1% 881.0 67.7% 677.0 72.0% 720.0

1000 85.9% 859.0 89.5% 895.0 69.1% 691.0 69.1% 691.0

1500 86.3% 1294.5 90.2% 1353.0 67.1% 1006.5 66.5% 997.5

1000 80.6% 806.0 88.7% 887.0 62.3% 623.0 69.1% 691.0

1000 85.6% 856.0 90.4% 904.0 66.7% 667.0 69.4% 694.0

1000 82.2% 822.0 90.7% 907.0 61.8% 618.0 61.5% 615.0

10000 8408 8891 6607 6784

SAGITEC

DELOITTE FAST ENTERPRISES GEOGRAPHIC SOLUTIONS SAGITEC

DELOITTE FAST ENTERPRISES GEOGRAPHIC SOLUTIONS

TOTAL POINTS AWARDED

DASPS-2232-19 PROCUREMENT EVALUATION COMMITTEE SCORING SHEET

ROUND 3

PRELIMINARY SUBMISSIONS

INTRODUCTIONS, PROJECT SUMMARY AND SOLUTION OVERVIEW

SYSTEM DEMONSTRATIONS – TAX

DAY 1 TECHNICAL DISCUSSIONS AND DEMONSTRATIONS

ORGANIZATIONAL CHANGE MANAGEMENT, PROJECT MANAGEMENT, AND TRAINING

SYSTEM DEMONSTRATIONS – BENEFITS

SYSTEM DEMONSTRATIONS – GENERAL


OVERALL PERFORMANCE AND POTENTIAL FIT

Based upon scoring results from Round 3 DAS PS and OED determined to move the two top-ranking Responsive Proposers to Round 4 Evaluation (“Round 4”). The Round 4 finalists were FAST Enterprises and Deloitte Consulting. DAS PS and OED determined not to carry over Round 3 scores to Round 4.

DAS PS and OED subsequently issued RFP Addenda 11 and 12 through which those parties released Round 4 evaluation requirements. Copies of Addenda 11 and 12 are included in this Notice as documents embedded in Attachment 2 to the Notice. The Committee has concluded Round 4 Scoring. FAST Enterprises prevails as the top-ranking Proposer for contract award. Round 4 scoring is included as an embedded document in the Notice at its Attachment 3.

A Proposer who claims to have been adversely affected by the selection of a competing Proposer shall have seven (7) calendar days from the date of this Notice in which to deliver a signed protest to DAS PS through the SPC. The SPC will deliver the Notice to all Proposers today via ORPIN (Oregon Procurement Information Network) release or individual emails to the Proposers’ named representatives, or both. Aggrieved Proposers have until 5:00 p.m. Pacific Time, on Thursday, September 17, 2020, to submit a protest. The submitted protest must bear the signature of a company official authorized to represent Proposer’s interests. Proposers may use an electronic signature copy via email to satisfy the protest submission deadline, provided they mail the original protest to the undersigned SPO analysts postmarked no later than the protest submission deadline.

To be adversely affected, a Proposer must demonstrate that all higher ranked Proposals were ineligible for selection. The State will not consider a protest submitted after the deadline. Protests will be resolved according to Oregon Administrative Rules.


DAS PS will address all protests that are timely submitted in accordance with ORS 279B.410. Within a reasonable time following DAS PS’s receipt of any protests, DAS will issue a written decision to the Proposer who submitted the protest. DAS PS will not consider protests that do not include the information required under ORS 279B.410.

DAS PS appreciates the time, resources and effort that all Proposers expended in responding to the comprehensive submission requirements throughout the long course of this procurement. On behalf of the State of Oregon, DAS PS encourages all Proposers to refer to ORPIN for future business opportunities with the state.

Please contact us with any questions regarding this decision.

Nicholas Betsacon, IT Procurement Strategist & Single Point of Contact

Oregon Department of Administrative Services

Enterprise Goods & Services

Procurement Services

1225 Ferry St. SE.

Salem, OR 97301-4285

Phone: (503) 267-6636

E-mail: [email protected]


ATTACHMENT 1 TO NOTICE OF INTENT TO AWARD IN RFP DASPS-2232-19

ROUND 3 DEMONSTRATION REQUIREMENTS

No. SCORED COMPONENT POINTS

1 PRELIMINARY SUBMISSIONS

Due 1 Business Day before first scheduled demonstration. 1,500

2 INTRODUCTIONS, PROJECT SUMMARY AND SOLUTION OVERVIEW 500

3

SYSTEM DEMONSTRATIONS – TAX

a. Employer Registration and Determination

b. Employer Account

c. Payroll Reporting

d. Tax Audit and Employer Accounting

e. Customer Portal

f. Paid Family and Medical Leave Insurance (PFMLI)

1,500

4


a. Rules

b. Data

c. Security

d. Integration/Interfaces

e. Authentication and Authorization

1,000

5 ORGANIZATIONAL CHANGE MANAGEMENT, PROJECT

MANAGEMENT, AND TRAINING 1,000

6

SYSTEM DEMONSTRATIONS – BENEFITS

a. Claim Intake

b. Weekly Claims

c. Adjudication

d. Workflow

e. Appeals

f. Benefit Payments

g. Benefit Extensions

h. Charges

1,500


i. Investigations

j. Special Programs

k. Customer Portal

7

SYSTEM DEMONSTRATIONS – GENERAL

a. Correspondence

b. Customer Portal

c. Recovery

d. Financial and Financial Reporting

e. Reporting

f. Research

1,000

8


a. New Program

b. Audit and Logging

c. System Accessibility, Capacity and Performance

d. System Maintenance and Support; System Updates

e. Third‐Party Applications

1,000

9 OVERALL PERFORMANCE AND POTENTIAL FIT 1,000

TOTAL POINTS 10,000



RFP ADDENDA 11 & 12 RE: ROUND 4 REQUIREMENTS

Addendum_11_OED_UI_MOD_RFP_03.20.2

Addendum_12_OED_UI_MOD_RFP_04.09.2



SCORING RESULTS FROM ROUND 4

RFP DASPS-2232-19_Rou

1

03-20-2020_Addendum #11 to RFP # DASPS-2232-19 OED UI MODERNIZATION | Start of Round 4

Evaluation: Site Visits, Maturity Assessments and Contract Redlines

ADDENDUM #11 FOR RFP # DASPS-2232-19 (Unemployment Insurance Modernization)

Effective today, March 20, 2020, the purpose of this Addendum #11 (“Addendum”) to Request for

Proposal DASPS-2232-19 (“RFP”) is to announce the start of Round 4 Evaluation (“Round 4”) through

which State will determine which of the two Finalists, Deloitte or FAST, presents the likely best fit for

Contract Award. The Addendum presents a high-level description of all aspects of Round 4 and

provides a detailed description of the first three components: Site Visits, Maturity Assessments, and

Contract Redlines. State will issue subsequent Addenda to reflect detailed requirements for the

remaining Round 4 components.

A. ROUND 4 EVALUATION OVERVIEW Through Round 3 Evaluation (“Round 3”) State chose the two Finalists who presented the likely best fit

for a single Contract Award – Deloitte and FAST. In Round 4 Evaluation (“Round 4”) State will determine

which one of the two Finalists is the actual best fit to meet the needs expressed in the procurement.

State will not carry over Round 3 points to Round 4.

Round 4 will include multiple evaluation components, some of which will generate scored assessment

by tallied point totals, and others that will serve to draw relative comparisons between the two Finalists

in the context of appropriate industry best practices and standards. Concerning the latter form of

assessment, the Evaluation Committee may convert comparisons to points on the basis of pre-

determined point allocations; i.e., the more favorable comparison results in a larger share of any

available points. The entire Round 4 process will facilitate State’s ultimate determination as to which

Finalist presents the actual best fit for Contract Award based upon a combination of point totals and

more favorable comparisons.

In Round 4, State will measure the following components (not necessarily in the listed order), and may

add more as appropriate in reaction to evaluation results:

1. Site Visits to States where Finalists have Solutions in Operation (“Site Visits”);

a. Detailed in this Addendum.

2. Maturity Assessment;


3. Contract Redlines Assessment;


4. Revised Price Proposals;

a. Detailed in subsequent Addenda.

b. Will include a requirement for submission of a high-level approach, plan, schedule and

related cost to incorporate Paid Family Medical Leave Insurance (“PFMLI”) tax

components into the proposed Solution. State will reserve a contractual option, with no

guarantee to its UI Modernization Contractor, to require the Contractor to implement

PFMLI tax components into its proposed Solution.

c. Finalists may access more information about the PFMLI opportunity at:

https://www.oregon.gov/employ/PFMLI/Pages/default.aspx

5. Fiscal Stability of Finalists;

a. Detailed in subsequent Addenda; and

6. Finalist Follow-up.

a. Detailed in subsequent Addenda.


2



b. Possible discussions with, and presentations from, each Finalist to address relevant

matters, from State’s perspective, discovered through the course of other Round 4

activity.

B. VIRTUAL SITE VISITS

Each Finalist included with its Proposal submissions a list of client references for whom the Finalist

implemented and operates or will soon operate completed Solutions comparable to the Solution State

seeks to acquire in the current procurement. In RFP Section 3.4.2, State declared that in the Final

Procurement Phase it would conduct “Site Visits” for certain Finalist client references.

State will now commence these Site Visits for the following client references listed by Finalist and

Organization Visited. The Evaluation Committee representatives will conduct “virtual” Site Visits, as

appropriate in the face of the current, declared Coronavirus Disease 2019 (COVID-19) Pandemic; and

may take up to several days to complete each visit. State will attempt to complete all Site Visits by

April 17, 2020.

FINALIST ORGANIZATION VISITED

Deloitte New Mexico

Deloitte Massachusetts

FAST Washington

FAST Michigan

The Site Visits are broad, deep extensions of the client reference check process. State will check all

listed client references for each Finalist, but will limit Site Visits to the above-listed organizations. In

the context of scope, maturity and character; the Site Visit selections for each Finalist are as

comparable as possible.

Through these Site Visits, the participating members from the Evaluation Committee will hold

unrecorded virtual meetings, using audio-video conference technology as appropriate, with their

logical counterparts from the Organizations Visited for discussions and examination of relevant

documentation; all from the perspective of the seven inquiry sections described in Attachment 1 to the

Addendum, Site Visits (“Attachment 1”); which attachment by this reference is incorporated into the

Addendum. The maximum possible score for the Site Visit component in Round 4 is 10,000 Points. The

point total is based on the sum of two Site Visit Final Rating Scores (up to 5,000 Points per Site Visit)

for each Finalist; with possible upward or downward scoring adjustments, as appropriate, based upon

outcomes from all other client reference checks and related Finalist Follow-up. The applicable

assessment and scoring criteria for Site Visits are set out in Attachment 1.

The Evaluation Committee members will not seek to communicate or otherwise interact with any

Finalist personnel during the course of these Site Visits, and Finalists must ensure that their personnel

will not seek out these participating members for any type of interaction.

C. MATURITY ASSESSMENT

In Addendum 7 to the RFP, State required Round 3 Proposers to confirm in writing their commitment

to comply with upcoming Round 4 requirements, including “independent organizational and process

maturity assessment and validation of code quality”. Deloitte and FAST submitted the required written

confirmation.

3



State has amended its contract with CSG, the independent quality management services consultant for

OED’s Modernization Program, to include CSG’s obligation to conduct the desired “Maturity

Assessment” for each Finalist on State’s behalf.

A redacted copy of the executed amendment is embedded in Attachment 2 to the Addendum, Maturity

Assessment (“Attachment 2”), which attachment by this reference is incorporated into the Addendum.

Attachment 2 sets out the Tasks, Deliverables and Acceptance criteria related to the work that CSG will

perform in connection with the desired Maturity Assessments. The Finalists must cooperate with the

process as appropriate.

D. CONTRACT REDLINES

Under RFP Section 5.4.1, Negotiation, State declares that “after” selection of a successful Proposer,

State Parties may enter into Contract negotiations with the successful Proposer. Moreover, by

submitting a Proposal, each Proposer agreed to comply with the requirements of the RFP, including

the terms and conditions of the Sample Contract (Attachment A-1), but subject to negotiation as set

forth below:

It may be possible to negotiate some provisions of the final Contract;

however, State Parties are not required to make any changes and

many provisions cannot be changed. Proposer is cautioned that the

State of Oregon believes modifications to the standard provisions

constitute increased risk and increased cost to the State. While State

Parties expect substantial negotiation of the Statement of Work, and

although many terms of the agreement are generally negotiable, State

Parties expect Proposer to base its proposal, including pricing, based

on the terms of the attached Sample Contract, including expectations

for indemnity, limitations of liability, and insurance requirements.

State Parties are willing to negotiate all items, except those listed

below:

a. Choice of law

b. Choice of venue

c. Constitutional requirements

d. Requirements of applicable federal and State law

Any subsequent negotiated changes are subject to prior approval of

the Oregon Department of Justice. In the event that the parties have

not reached mutually agreeable terms within thirty (30) calendar days

from resolution of any Award Protest(s), State Parties, at their

discretion, may terminate Negotiations and commence Negotiations

with the next highest ranking Proposer. (Bold, underlined emphasis

added).

Rather than delay the start of contract negotiations until after selection of one of the Round 4 Finalists

for Contract Award, State desires to expedite the negotiation process by requiring each Round 4 Finalist

to provide Sample Contract “Redlines” within two weeks from the effective date of the Addendum.

Accordingly, both Finalists must submit their Contract Redlines via email to the Single Point of Contact

(“SPC”) by Monday, April 6, 2020, no later than 5:00 p.m. Pacific Time.

Subject to the four exceptions listed in RFP Section 5.4.1.a. to d., the Round 4 Finalists may either

submit the Sample Contract in its entirety with redlines; or a separate document setting out just those

negotiable redlined Sample Contract provisions, exhibits and attachments that Proposers wish to

negotiate with State. Regardless of the manner of submission, Sample Contract Redlines must clearly

show deletions from, and insertions to, the affected Sample Contract provisions, exhibits and

attachments, e.g., strikethrough text for deletions and bold underlined text for insertions.

4



State will award 300 additional points for submitted Sample Contract Redlines in Round 4. Failure to

submit meaningfully complete Contract Redlines will result in no award of additional points. In

Attachment 3 to the Addendum, Sample Contract Redlines Submission, which by this reference is

incorporated into the Addendum; State sets out its expectations concerning attributes of Contract

Redlines.

E. CONCLUSION

Nicholas Betsacon remains as the assigned SPC for this procurement.

Nicholas Betsacon, IT Procurement Strategist



1225 Ferry St. SE.

Salem, OR 97301-4285

Phone: (503) 267-6636


-END OF DOCUMENT (Except for Addendum Attachments 1 to 3, which follow)-

mailto:[email protected]

5



Addendum No. 11 to RFP # DASPS-2232-19

ATTACHMENT 1

VIRTUAL SITE VISITS

Purpose:

This document serves as a guide for the collection of pertinent site information as part of the

Unemployment Insurance Modernization Round 4 Evaluation.

Completion Instructions:

This document is divided into seven sections as follows. Questions listed in a section are intended as

suggested initial questions for Scorers to pose for the purpose of starting thoughtful and thorough

discussions with the Organization Visited about the section. State’s intent is to reasonably exhaust its

inquiry for relevant information pertaining to each section. Therefore, Scorers are not limited to the

suggested initial questions in a section, and may use their reasonable discretion to determine the order

of questioning and the nature of related follow-up questions and requests for access to relevant

documentation.

1. Section 1, Vendor Relationship: These are questions related to the client state’s overall

business relationship (e.g., vendor management) and satisfaction with the Proposer.

2. Section 2, Implementation: These are questions related to the client state’s experience with

the Proposer during the project, including but not limited to: actual versus planned budget,

schedule, and scope; experiences with design, development, testing, training, data conversion,

change management; project staffing; etc.

3. Section 3, Maintenance and Support: These are questions related to the client state’s

experience with the Proposer relating to maintenance and support, including but not limited to

costs, timeliness and responsiveness, flexibility, quality, and roles and responsibilities (e.g.,

extent to which the client state is dependent on the Proposer for ongoing support) of upgrades,

enhancements, and system modifications; and the simplicity or complexity of architecture and

its relation to maintenance and support.

4. Section 4, Contract: These are questions related to the client state’s experience with the

Proposer relating but not limited to contract negotiations, contract administration, and change

orders.

5. Section 5, Solution Overview: These are questions related to the client state’s use of the

Proposer’s system, including but not limited to system architecture; integration with other

systems, both internal and external; components and functionality provided by the solution and

third-party applications; functionality performed, or auxiliary aides used, outside of the system

or by third-party applications; and the client state’s adherence to the Proposer’s recommended

solution.

6. Section 6, End-user Experience: These are questions related to the overall experience and

satisfaction of the client state’s users with the system.

7. Section 7, Lessons Learned: These are questions related to overall lessons learned from the

client state including but not limited to what went well, what they would do differently, and

project closeout reports.

Each section contains four columns (ID, Question, Rating, and Notes), and the table on the next page

(“Table 1”) provides a definition for each column. Scorers may use the “Notes” column to document

observation details. The only entry required to be filled out in the Rating column is the “Final Rating”,

which will be the result of a collaborative effort by the participating Scorers as reflected in Table 1.

6



Table 1 Definitions and Instructions for filling out the Onsite Client Visit Evaluation Document

ID This is just to provide a reference point.

Question

Describes the question or requirement that the evaluator will rate. There can be one or

more related questions attached to one requirement. Example: How easy is it to

perform this search and how easy is it to generate reports?

Rating

For Sections 1 through 7: The Scorers are to collaborate on a final rating for each

section on a scale between one (1) and five (5), with one (1) being the lowest and five

(5) being the highest, in how well the product/Proposer meets the

question/requirement. Scorers may rate in decimal fractions, e.g., 1.25, 2.50, 3.75, and

so on.

Each final rating number equals a quantum of points, e.g., If 1 rating number equals

100 points a final rating of 2.75 equals 275 Points. Sections may be weighted

differently depending upon the points designated for each rating number in a particular

section. The maximum possible score for the Site Visit component in Round 4 is

10,000 Points. The point total is based on the sum of two Site Visit Final Rating Scores

(up to 5,000 Points per Site Visit) for each Finalist; with possible upward or downward

scoring adjustments, as appropriate, based upon outcomes from all other client

reference checks and related Finalist Follow-up.

The Scorers will divide into single-member or tandem teams, with each team focusing

on logical counterparts from the Organization Visited. In each instance, as appropriate,

teams will conduct their discussions in virtual meetings with these logical counterparts

in the context of all seven sections. It may take several virtual meetings conducted

over a period of several days to complete a Site Visit.

As appropriate, during, and certainly following, Site Visits, all Scorers will share their

collected information and collaborate to establish a Final Rating for each Site Visit for

each Finalist.

State will also use, as appropriate, collected information from Site Visits and other

client reference checks in connection with the Finalist Follow-up component in Round

4. For example, Finalists will have an opportunity to address negative information

generated through the course of Site Visits and other client reference checks.

Notes

This is space for Scorers to use for notes or to annotate any feature or concern item(s)

and/or to document observations to the demonstrated event (e.g.: System appeared

intuitive and easy to navigate, system failed to function properly when attempting a

specific task… etc.).

Scorers’ Names and Titles, Organization Visited, and date of visit

Table 2: Site Visit Information

Scorer Name and Title

Proposer Name

Client Site/Agency

Date of Visit

7



Section 1, Vendor Relationship (Each Final Rating Number = 100 Points)

ID Question

Rating

1 to 5 Notes

Describe how it was working with the Proposer over the course of the project.

Describe how it has been working with the Proposer post implementation.

Describe how well the Proposer and their solution met your expectations and requirements.

Did the Proposer follow the approach and processes stated during the procurement process during the actual project and post implementation? Please describe.

Describe how well the Proposer worked with other vendors or parties involved during the project.

Are there any other comments, lessons learned, or advice that will help us succeed?

Final Rating

8



Section 2, Implementation (Each Final Rating Number = 250 Points)

ID Question

Rating

1 to 5 Notes

Describe how satisfied you are with the Proposer’s overall implementation approach.

What, if any, were the major issues that arose during the project?

a) What was the root cause? b) How were they resolved? What was the Proposer’s

role in resolving the issue?

Did you experience any major changes to the scope during the project?

a) If yes, what were those changes and why did they occur?

b) Describe the Proposer’s response to the scope change. How helpful was it in reaching a feasible and effective resolution?

How was scope managed and planned into sprints or iterations?

Did you experience any major changes to the schedule during the project?


b) Describe the Proposer’s response to the schedule change. How helpful was it in reaching a feasible and effective resolution?

Did the Proposer’s schedule allow for an appropriate amount of time for each phase (e.g. design, development, testing, training, etc.)?

Did you experience any major changes to the budget during the project?


b) Describe the Proposer’s response to the change. How helpful was it in reaching agreement?

9



How were requirements managed from inception through testing and delivery? How helpful was this approach, and is there anything you would do differently next time?

a) Were any requirements missed until late in the project?

a) If yes, what were those requirements? How did the Proposer resolve this gap?

How well did the Proposer track, manage, and communicate project components?

a) What tools (e.g., software) did the Proposer use, and did agency staff have access to these tools? Did you use additional tools to manage the project?

How much customization did you seek for the system to meet your state’s requirements?

a) Were there specific areas that required more customization than others? If yes, please describe those areas and the customization needed.

b) Why was the Proposer’s solution impractical without customization?

c) Was customization essential to meet law and rule, or was it done in order to support established ways of doing business?

Did the Proposer appropriately staff the project?

a) If yes, please describe the approach. b) If no, what would you like to have seen done

differently? What were the impacts?

Were the Proposer’s staff available and responsive?

What was the Proposer’s approach to testing?

a) How involved were your staff during testing?

How do you define “stabilization”? Did you have a formal stabilization period, and what was the experience like? How long did it take?

What was the training approach for field staff and central staff, including support provided? How effective was the training? What made it effective or ineffective, and what would you do differently?

How long did it take average users to become comfortable in the system? For those who struggled, why and how would you better support them next time? What factors contributed to the success or challenges in this experience?

10



What are some of the critical success factors you see that a state needs to ensure staff are prepared to accept and support change?

What challenges and successes did you encounter in planning, developing, or implementing interfaces?

a) What was the Proposer’s role in those challenges and successes?

What challenges and successes did you encounter around data conversion, sharing or use of data, and reporting?

a) What was the Proposer’s role in those challenges and successes?

Do you have any other insights or advice that will help us succeed at implementation, or provide important insights into working with the Proposer?

Final Rating

11



Section 3, Maintenance and Support (Each Final Rating Number = 250 Points)

ID Question

Rating

1 to 5 Notes

Describe your system maintenance and administration processes.

a) What is the Proposer’s responsibility for supporting the system?

b) What is the state’s responsibility?

How many internal IT staff are dedicated to supporting the system?

a) Do you feel this is sufficient? b) Are there more or less staff than you originally

anticipated needing to support the system? c) What are the specific roles and responsibilities of

your internal staff? d) Is there end-user (non-IT staff) involvement to

support the system? If so, what tasks do they perform? For example, do business analysts update business rules?

To what degree can your staff make system changes (e.g., system values, business rules, etc.) compared to relying on the Proposer?

Is this according to your wishes and your design? Did you have options?

Describe the timeliness and capacity of the Proposer to support the system (e.g., break fixes, enhancements, minor and major upgrades). For instance, how quickly are changes (such as those due to state legislative or federal mandates) implemented?

Describe how the Proposer has kept system components, including third-party components, updated and current.

Have you gone through a major upgrade?

a) If so, how was implementation? b) What was the Proposer’s role and what support did

they offer? c) What, if anything, would you do differently next

time?

Are training materials updated along with system updates?

a) If yes, describe how that information is provided and is that training work completed by the Proposer or your staff?

How do you manage tickets? Do you use the Proposer’s system or another system? Can the agency track and report on these issues easily including response times?

12



Describe your system’s stability and performance (scheduled and unscheduled downtime).

a) Describe the Proposer’s responsiveness to stability and performance issues.

What are your service-level agreements (SLAs)?

How do you manage your service levels?

Final Rating

13



Section 4, Contract (Each Final Rating Number = 50 Points)

ID Question

Rating

1 to 5 Notes

Describe how initial contract negotiations went.

a) Do you have any advice or lessons for us with regards to contract negotiation?

How long did contract negotiations take?

Have you renewed the contract?

a) If yes, how was the process? b) Were there any significant changes from the initial

contract? c) Why were they needed (or not?) d) If no, what changes would you like to see from your

initial contract?

Have you experienced any amendments or change orders?

a) If yes, what were they and why? b) What were the associated costs?

Have you had any significant contractual issues either during the execution phase and/or during operations and maintenance? What have these issues been, and how would you advise us to avoid them?

What are your vendor management metrics?

Is the Proposer’s performance in accordance with the contract requirements?

a) If no, in what areas? b) What measures were taken to remedy the

performance?

Final Rating

14



Section 5, Solution Overview (Each Final Rating Number = 200 Points)

ID Question

Rating

1 to 5 Notes

What efficiencies have you gained since implementation?

How has the customer experience changed since implementation?

Was any functionality missing or delayed at go-live? a) Is that functionality now present? If not, why? What

was the Proposer’s role in implementing or not implementing that functionality?

b) Is there functionality missing from the system that requires ancillary systems or workarounds?

Were any defects present at go-live? a) If so, what were they? b) How did the Proposer resolve, or propose to

resolve, those defects?

What is the level of automation in the new system compared to the "legacy" system?

a) Has there been any challenges as a result of automation (due to business process changes, staff adaptability, etc.)?

What benefits or goals did you expect to achieve prior to implementation and were they met?

How did you measure success?

a) Did you develop metrics? Quantitative, qualitative, or both? Are you willing to share them?

Did you have to change any policies, rules, or statutes to accommodate new processes?

Describe any unanticipated decreased productivity at go-live (e.g., customer call volume increased due to questions on using the system, staff processing/workflows slower than anticipated, system performance slow, etc.).

Final Rating

15



Section 6, End-user Experience (Each Final Rating Number = 100 Points)

ID Question

Rating

1 to 5 Notes

What do the users of the system like best about the new system and why? How does this vary across different types of users such as claimants, employers, Tax staff, Benefits staff, IT staff, partners, etc.?

What do users like least about the new system and why?

Describe the feedback received from your customers after go-live.

a) Do you have any measures of customer satisfaction? b) If so, please describe.

Describe the feedback received from your staff after go-live.

a) Do you have any measures of staff satisfaction? b) If so, please describe.

Has your staffing model changed as a result of your new system?

a) If so, how? b) What was your experience of the transition? What,

if anything, would you do differently? c) What are the successes and challenges of the new

model?

How has the new system affected staff performance? a) Do you have any measures of staff efficiency and

effectiveness? b) If so, please describe.

Describe the feedback received from your partners after go-live.

What was the learning curve like for end users?

Final Rating

16



Section 7, Lessons Learned (Each Final Rating Number = 50 Points)

ID Question

Rating

1 to 5 Notes

Describe what went well during the project and how the Proposer contributed to that success.

a) What has gone well since implementation and how has the Proposer contributed to that success?

What challenges did you experience during the project? a) What role did the Proposer play in those

challenges? b) What were your lessons learned?

What challenges have you experienced post go-live? a) What role did the Proposer play in those

challenges? b) What were your lessons learned?

What were the main sources of frustrations on your project?

Were there any surprises, in general and regardless of whether good or bad, over the life of the project?

a) If yes, please describe those surprises.

Would you share any staffing or vendor management lessons learned?

a) If yes, please describe the lessons learned.

What were the primary impacts or changes you experienced during your modernization effort?

If you had to start all over again, would you select the same Proposer? Why or why not?

If you could start over from the beginning, what would you do differently?

Final Rating

-END OF ATTACHMENT 1-

17




ATTACHMENT 2

MATURITY ASSESSMENTS

Amendment 1 to

OED-CSG iQMS WOC.DASPS-1620-19_REDACTED FOR ADDENDUM 11.pdf -END OF ATTACHMENT 2-

18




ATTACHMENT 3

SAMPLE CONTRACT REDLINES SUBMISSION

The primary purpose for this component of the evaluation is to help the State ease and expedite the

contract negotiation process with the Finalist selected for Contract Award.

The State values transparent and straightforward business relationships in which each party clearly

conveys their interests and concerns, so that we may amicably and efficiently work towards agreement.

If at any time between issuance of intent to award and final award State determines that a Finalist is

not negotiating in a manner consistent with these values; then State Parties, at their discretion, may

terminate Negotiations with that Finalist, and commence Negotiations with the next highest-ranking

Proposer – starting with the other Finalist in Round 4 and then possibly moving, as necessary and

appropriate, to the number 3 and number 4 Proposers from Round 3.

The Sample Contract Redlines Submission must clearly indicate proposed modifications to negotiable

Sample Contract provisions, exhibits, and attachments as prescribed in Addendum #11 to RFP DASPS-

2232-19, and within the context of the parameters for good faith negotiation set out in the RFP in its

Section 5.4.1, Negotiation.

According to such section in the RFP, State has marked for negotiation every part of the Sample

Contract, except for those few provisions related to: Choice of Law, Choice of Venue, Constitutional

Requirements and Requirements imposed by applicable federal and State laws. While State Parties

expect substantial negotiation of the Statement of Work, and perhaps Service Levels; State wants to

avoid having to negotiate:

a. Excessive supplemental terms and conditions;

b. Extensive changes to State’s standard terms and conditions (State believes

modifications to the standard provisions constitute increased risk and increased cost to State); or

c. Unnecessary material changes to critical terms and conditions.

State will award 300 Points for submission of meaningfully complete Contract Redlines if the Finalist

provides a Sample Contract Redline (including all exhibits and attachments with proposed redlines)

that represents the Finalist’s initial good faith negotiation of the Sample Contract; and 0 Points if the

submission fails to do so. A meaningfully complete submission should address all significant and

foreseeable concerns of the Finalist. Finalists must include specific requested insertions and deletions,

and are encouraged to also provide detailed explanations (in plain language, alongside the contract

language) explaining the concerns it seeks to address through each proposed modification.


Amendment No. 1 to Work Order Contract No. DASPS-1620-19 DAS Procurement Services, Version 1.0 – February 20, 2019 Page 1 of 12

Amendment No. 1 to Contract No. DASPS-1620-19 A Work Order Contract under

Master Price and Services Agreement No. 8470 with CSG Government Solutions Independent Quality Management Services for the

Oregon Employment Department’s Modernization Program This is Amendment No. 1 to Work Order Contract No. DASPS-1620-19, dated May 17, 2019, as amended from time to time (“Contract”) between the State of Oregon, acting by and through its Department of Administrative Services, Procurement Services (“DAS”) on behalf of the Oregon Employment Department (“Authorized Purchaser” or “OED”), and CSG Government Solutions, Incorporated, an Illinois Corporation (“Contractor”). This Amendment is effective on the date signed by all parties and upon receipt of all approvals necessary for signing (“Amendment Effective Date”).

RECITALS

1. This Amendment No. 1 (“Amendment”) documents adjustments to selected Contract terms and conditions and selected exhibits to the Contract, as follows:

(a) Section 3.3 in the Contract, Delivery and Review of Deliverables, Subsection 3.3.3;

(b) Exhibit 1 in the Contract, Initial Statement of Work; and

(c) Exhibit 3 in the Contract, Consultant And Authorized Purchaser Key Personnel For WOC.

2. The Amendment preamble and these recitals assert relevant facts and circumstances to which the parties agree; and these recitals set forth parentheticals with certain acronyms and shortened terms in quotes. The parties will use these acronyms and shortened terms as appropriate references throughout the Amendment.

3. Authorized Purchaser, DAS, and Contractor executed the Contract effective May 17, 2019. The Contract is the result of a “best value” opportunity issued under Request for Quote (“RFQ”) #OED-0129-19. The RFQ documented a portfolio of projects opportunity for Independent Quality Management Services (“QMS”) related to Authorized Purchaser’s Modernization Program. Contractor prevailed in the best value opportunity for Contract award.

4. Through this Contract, Contractor provides QMS for Authorized Purchaser’s Modernization Program. The Contract’s initial Statement of Work, Contract’s current Exhibit 1 (“SOW”), focuses exclusively on QMS for the first project released in the Modernization Program, Authorized Purchaser’s Unemployment Insurance (“UI”) Modernization Project (“Project”). In the context of the Project, Authorized Purchaser and DAS have released Request for Proposals DASPS-2232-19 (“UI Mod Procurement”), through which Authorized Purchaser seeks to procure an integrated UI solution that supports all functions and processes of Authorized Purchaser’s UI program for tax and benefits.

5. The UI Mod Procurement established two evaluation phases: Initial Procurement Phase, comprised of two rounds through which the Authorized Purchaser and DAS evaluate Responsive Administrative and Technical Proposals to establish a Competitive Range of Proposers; and the Final Procurement Phase, comprised of at least two rounds through which


Competitive Range Proposers travel to Authorized Purchaser’s facility in Salem, Oregon to demonstrate operation of their respective UI solutions in the context of particular requirements and factual scenarios, and Authorized Purchaser sends representatives to inspect and assess particular out-of-state sites (“Travel Sites”) where the two top-ranking Competitive Range Proposers (“Finalists”) have implemented operational integrated UI solutions.

6. The Final Procurement Phase in the UI Mod Procurement also requires the Finalists to cooperate with Authorized Purchaser in permitting “Maturity Assessments” of their respective UI solution operations (“UI Operations”) at particular Travel Sites. These Maturity Assessments call for independent examination of particular aspects of the Finalists’ UI Operations at the affected Travel Sites.

7. This independent examination must be conducted in the context of detailed assessment checklists drawn from relevant criteria within the appropriate standard established by the International Organization for Standardization (“ISO”); an international standard-setting body composed of representatives from various organizations promoting worldwide proprietary, industrial, and commercial standards. The appropriate standard for the desired Maturity Assessments is the newest version of the international standard related to assessing software lifecycle processes, ISO/IEC/IEEE 12207:2017, Systems and Software Engineering – Software Life Cycle Processes (“ISO 12207”).

8. Contractor has the relevant experience and expertise needed to conduct the desired Maturity Assessments in the context of ISO 12207. These assessments are within the Contract’s scope as a combination of Risk Assessment, Quality Control and Quality Assurance Deliverables. Contract’s SOW includes the following five Tasks, each with at least one related Deliverable:

(a) Task 1: Risk Assessment (Risk Assessment Reports);

(b) Task 2: Quality Management Planning (Quality Checklists and Special Quality Control-related Requests);

(c) Task 3: Quality Control (Quality Status Reports and Quality Control Review Reports);

(d) Task 4: Quality Assurance (Quality Assurance Status and Improvement Reports); and

(e) Task 5: Dynamic Quality Control via Independent Solution Testing (Master Testing Plan, Testing Results Reports).

The desired Maturity Assessments combine elements from all five Task areas, and most logically call for treatment as Special Quality Control-related Deliverables under Task 2, Deliverable 2.5.

9. Contract’s SOW includes a section, which sets out Contract’s Deliverables and Payment Schedule (“DPS”). Through the DPS, the parties identify Deliverables in a matrix by number, title, due date and cost (“DPS Matrix”). The parties have revised the current DPS Matrix to account for Deliverable modifications and additions to which the parties agree in this Amendment (including a previous Contract Change Order the parties executed, a copy of which is attached to the Amendment as Attachment 2). The revised DPS Matrix is attached to this Amendment as its Attachment 1.

10. Contractor is required to provide periodic Quality Status Reports under Task 3 (Deliverable 3.3). Authorized Purchaser bargained for up to three Quality Status Reports per quarter with one


delivered at the end of each month in a quarter, except for the last month in the quarter unless otherwise mutually agreed. Authorized Purchaser did require the delivery of three Quality Status Reports in the first quarter during which Quality Status reporting commenced, but now desires to explicitly reduce the number of periodic Quality Status Reports due in a quarter from three to two.

11. Contractor must expend the same level of effort in a quarter to keep abreast of Project activity for the purpose of developing and submitting acceptable Quality Status Reports, regardless of the number of report instances required in a quarter. The current cost per report instance ($19,000.00 based upon up to 104 hours of effort calculated at an hourly rate of approximately $182.69) is premised on Contractor developing and delivering three such reports per quarter. Contractor requests an increase in cost per report instance for those quarters in which Authorized Purchaser requires only two Quality Status Reports. The agreement of the parties regarding cost increase per report instance under the circumstances described in Recital No. 10 and this Recital No. 11 is reflected in the Amendment’s Attachment 1.

12. Authorized Purchaser desires to add a Deliverable 3.1, Quality Control Review Report concerning its Stakeholder Engagement Plan, which plan it will provide to Contractor in or about May 2020. The agreement of the parties regarding cost for the additional Deliverable is reflected in the Amendment’s Attachment 1.

13. Authorized Purchaser established a contingency amount of $200,000.00 under the Contract to apply to future QMS Deliverables (“Contingency”). The Contingency is already added into the Contract’s Maximum Payment Amount listed in Section 6.1 of the Contract. Although depicted in the DPS under Task 2, Deliverable 2.5, Authorized Purchaser may apply the Contingency to any in-scope QMS work under the Contract. Authorized Purchaser intends to apply Contingency funds to compensate Contractor for any Accepted additional Deliverables required under the Amendment. The cost for the new Maturity Assessment Deliverables is $104,400.00 ($52,200.00 for each Deliverable); and the cost for the additional Deliverable 3.1 for Quality Control review of Authorized Purchaser’s anticipated Stakeholder Engagement Plan is $20,720.00.

AMENDMENT

The Contract is amended as follows (as appropriate new language is indicated by underlining and bold and deleted language is indicated by strikethrough):

1. To correct an administrative error in Contract Section 3.3, Delivery and Review of Deliverables, Subsection 3.3.3, the parties amend Subsection 3.3.3 as follows:

If the Authorized Purchaser determines that a Deliverable does not meet the Acceptance Criteria in all material respects, Authorized Purchaser will notify Consultant in writing of Authorized Purchaser’s rejection of the Deliverable, and describe in reasonable detail in such notice the Authorized Purchaser’s basis for rejection of the Deliverable. Upon receipt of notice of non-acceptance, Consultant shall, within a fifteen (15) Business Day period, modify or improve the Deliverable at Consultant’s sole expense so that the Deliverable meets, in all material respects, the Acceptance Criteria, and notify the Authorized Purchaser in writing that it has completed such


modifications or improvements and re-tender the Deliverable to Authorized Purchaser. Authorized Purchaser will thereafter review the modified or improved Deliverable within fifteen (15) Business Days of receipt of the Consultant's delivery of the Deliverable. Failure of the Deliverable to meet the Acceptance Criteria in all material respects after the second submission will constitute a default by Consultant. In the event of such default, Authorized Purchaser may either: (i) notify Consultant of such default and instruct Consultant to modify or improve the Deliverables as set forth in this section 3.3.3, or (ii) notify Consultant of such default and pursue its remedies for default under Section 13 16 of this Contract.

2. To account for Deliverable modifications and additions to the Contract SOW and its DPS, the parties amend the SOW and its DPS as follows:

(a) Parties amend SOW at Task 2, Deliverable 2.5, to require Contractor to provide Maturity Assessments in the context of ISO 12207 according to the following requirements:

5. Deliverable 2.5: Special Requests

At the Authorized Purchaser’s request, and if not already required in connection with another QMS Deliverable, the Consultant may be required to appear before various committees or individuals to discuss the overall strategic direction and progress of the Project, or complete other “special” work. Special work includes independent analysis, recommendations on unforeseen problems, opportunities for improvement, research and recommendations on alternative courses of action, and additional risk management activities. In the instances requiring internal/external presentations, the Consultant should accompany the Authorized Purchaser’s staff to make presentations, help answer questions, and provide the Consultant’s assessment of confidence on the Project’s ability to satisfy its declared goals and objectives.

In response to a Deliverable 2.5 request, and within the time prescribed in the request, the Consultant shall provide the Authorized Purchaser with a written fixed cost proposal and schedule.

In the factual context of the Recitals from Amendment No. 1 to the Contract, especially those Recitals numbered 5 through 8, Authorized Purchaser requests Contractor to conduct and report on Maturity Assessments in the context of ISO 12207, and according to the process and requirements listed below and in the Deliverables and Payment Schedule Matrix as revised in Amendment No. 1 to the Contract; to all of which Contractor agrees and Contractor shall:

(a) Plan to conduct Maturity Assessments:

1) For each of two UI Mod Procurement Finalists during two-day site visits to one Travel Site per Finalist. Authorized


Purchaser will work with the Finalists to designate the Travel Sites.

2) Based on detailed checklists drawn from applicable and relevant ISO 12207 criteria (“Checklists”); and which incorporate, as Contractor deems necessary and appropriate, additional criteria from other relevant industry standards.

i. Checklists, and as appropriate the Assessment Plan and Assessment Report format, must include all relevant criteria needed to elicit information and data that will facilitate Authorized Purchaser’s confirmation of a Finalist’s capability to deliver an operational integrated UI solution that supports all functions and processes of Authorized Purchaser’s UI program for tax and benefits; including without limitation the following considerations:

1. Finalist Software Development Lifecycle methodology and process reviews and assessments;

2. Travel Site-related artifact reviews and assessments;

3. Assessment of information gathered through observations of Finalist Travel Site activities;

4. Development of Finalist interview questions; and

5. Concerning Finalists’ integrated UI solutions, distinguishing “Base” functionality from “Add-on”, i.e., Configuration versus Customization.

a. “Configuration” means the creation of a comprehensive collection of settings and values that are referenced by the software system in order to deliver required functionality. Configuration does not lead to a forked code base, and an existing configuration may be easily applied to a new or updated base system. Configuration approaches may for example include site code or configuration files that are used to maintain Oregon-specific


functionality across updates to the base system. (Note that even without forked maintenance, testing of the configured system will always be required whenever the configuration or the base system are changed.)

b. “Customization” means the modification of an existing base software system for the unique needs of OED in such a way that the changes require forked (separate and ongoing) maintenance and testing from that required for the base system, i.e. a forked or separate code base. Note that Contractor may elect to make changes to the base system in lieu of customization as a way of meeting State requirements; if done in such a way that maintenance and testing of the base system are not forked and the effort may be shared across multiple installations of the same code base, then OED will consider this to be Base rather than Add On.

3) In compliance with an Assessment Plan and Checklists, which Authorized Purchaser has reviewed and accepted (“Accepted Assessment Plan and Checklists”).

(b) Conduct Maturity Assessments in compliance with Accepted Plan and Checklists, and in the context of the Written Assessment Report format, which Authorized Purchaser has accepted.

(c) Develop and Submit to Authorized Purchaser Written Assessment Reports, one for each Finalist, in a format previously accepted by Authorized Purchaser, and which shall include without limitation:

1) Suggested weighting of the ISO 12207 and related criteria based on Contractor’s knowledge and on industry best practices. Authorized Purchaser will consider such suggestions for its informational purposes, only – not as Contractor’s recommended scoring of Finalists for Contract award.


(b) Parties reconfirm their agreement reflected in the Change Order attached to the Amendment as its Attachment 2, which attachment by this reference the parties incorporate into the Amendment.

(c) Parties amend the DPS Matrix in the SOW as reflected in Attachment 1 to the Amendment, Revised Deliverables and Payment Schedule Matrix; which attachment by this reference the parties incorporate into the Amendment.

3. To account for adjustments to Contractor’s Key Persons, the parties amend the Contract’s Exhibit 3, Consultant and Authorized Purchaser Key Personnel for WOC (“Personnel Exhibit”), as depicted in the revised Personnel Exhibit set out in Attachment 3 to the Amendment, which attachment by this reference the parties incorporate into the Amendment.

4. Except as expressly amended above, all other terms and conditions of Contract are still in full force and effect. Contractor certifies that the representations, warranties and certifications contained in the Contract are true and correct as of the Amendment Effective Date and with the same effect as though made at the time of this Contract.

Certification:

Any individual signing on behalf of Contractor has the authority and knowledge to make the following certifications, and hereby certifies under penalty of perjury:

(a) Contractor has no undisclosed liquidated and delinquent debt owed to the State or any department or agency of the State;

(b) Contractor complies with the provisions of ORS 652.220 and does not discriminate against any of Contractor’s employees in the payment of wages for work of comparable character, the performance of which requires comparable skills, or pay any employee at a rate less than another for comparable work, based upon sex. Additionally, Contractor declares that it does not discriminate in its employment practices with regard to race, creed, age, religious affiliation, gender, disability, sexual orientation, national origin; that when awarding subcontracts, Contractor does not discriminate against any business certified under ORS 200.055 as a disadvantaged business enterprise, a minority-owned business, a woman-owned business, a business that a service-disabled veteran owns or an emerging small business; and Contractor has a written policy and practice, that meets the requirements described in ORS 279A.112 (formerly HB 3060), of preventing sexual harassment, sexual assault and discrimination against employees who are members of a protected class.

(c) The number set forth in the contract is Contractor’s correct taxpayer identification number;

(d) Contractor is not subject to backup withholding because:

i. Contractor is exempt from backup withholding;

ii. Contractor has not been notified by the IRS that Contractor is subject to backup withholding as a result of a failure to report all interest or dividends; or


iii. The IRS has notified Contractor that Contractor is no longer subject to backup withholding.

(e) For a period of no fewer than six calendar years preceding the Amendment Effective Date, Contractor has faithfully complied with and is not in violation of:

i. All tax laws of this state, including but not limited to ORS 305.620 and ORS chapters 316, 317, and 318; and

ii. any tax provisions imposed by a political subdivision of this state that applied to Contractor, to Contractor’s property, operations, receipts, or income, or to Contractor’s performance of or compensation for any work performed by Contractor; and

iii. Any tax provisions imposed by a political subdivision of this state that applied to Contractor, or to goods, services, or property, whether tangible or intangible, provided by Contractor; and

iv. Any rules, regulations, charter provisions, or ordinances that implemented or enforced any of the foregoing tax laws or provisions.

(f) In the event that Contractor is a general partnership or joint venture, that Contractor signature(s) on this Amendment constitute certifications to the above statements pertaining to the partnership or joint venture, as well as certifications of the above statements as to any general partner or joint venturer signing this Amendment.

SIGNATURES COMMENCE ON NEXT PAGE


CONTRACTOR

CSG GOVERNMENT SOLUTIONS, INC.

STATE OF OREGON, acting by and through its DEPARTMENT OF ADMINISTRATIVE SERVICES, PROCUREMENT SERVICES

By: By:

Title: Title:

Date: Date:

FEID #

STATE OF OREGON, acting by and through its EMPLOYMENT DEPARTMENT

By: Approved pursuant to ORS 291.047

Title: By: JOHN F. (JACK) MCDONALD

Date: Senior Assistant Attorney General

VIA EMAIL DATED February 27, 2020

Recommended Approval from Enterprise Information Services (Office of State Chief Information Officer)

VIA EMAIL DATED ______________________


Amendment No. 1 to Work Order Contract DASPS-1620-19

ATTACHMENT No. 1

REVISED DELIVERABLES AND PAYMENT SCHEDULE MATRIX

REDACTED



ATTACHMENT No. 2

CHANGE ORDER RE: DELIVERABLE 3.1.5

REDACTED



ATTACHMENT No. 3

Revised Exhibit 3, Consultant and Authorized Purchaser Key Personnel for WOC

EXHIBIT 3

CONSULTANT AND AUTHORIZED PURCHASER KEY PERSONNEL FOR WOC

REDACTED

1 04-10-2020_Addendum #12 to RFP # DASPS-2232-19 OED UI MODERNIZATION Round 4: Revised Cost Proposal, PFML Planning & Cost, Fiscal Stability, Client References & Finalist Follow-up

ADDENDUM #12 FOR RFP # DASPS-2232-19 (Unemployment Insurance Modernization)

Effective today, April 10, 2020, the purpose of this Addendum #12 (“Addendum”) to Request for Proposal DASPS-2232-19 (“RFP”) is to provide more detailed information concerning the following Round 4 Evaluation (“Round 4”) components:

1. Scoring Completed Maturity Assessments;

2. Revised Cost Proposal Submission;

3. PFMLI Planning and Cost Submission;

4. Revised Fiscal Stability Submissions;

5. Client Reference Checks; and


A. CONTRACTOR RESPONSIBILITY

Please recall that through Round 4 State will determine which of the two Finalists, Deloitte or FAST, presents the likely best fit for Contract Award. A very important part of Round 4 is validation of Contractor Responsibility, i.e., demonstrated capability and capacity to carry out all contractual responsibilities as evidenced by:

1. Having appropriate financial, material, equipment, facility, and personnel resources and expertise, or having the ability to obtain the resources and expertise, necessary to meet all contractual responsibilities.

2. Completion of previous contracts of a similar nature with a satisfactory record of performance.

3. Having a satisfactory record of integrity.

4. Having the legal qualifications to contract with the Agency.

5. Compliance with the tax laws of the state.

6. Having a certification from DAS for training in pay equity provisions of state law.

7. Supplying all necessary information related to the determination of responsibility.

8. Having no record of being debarred by the State or Agency.

Addendum #11 presented a high-level description of all aspects of Round 4 and provided a detailed description of the first three components: Site Visits, Maturity Assessments, and Contract Redlines. State indicated it would issue subsequent Addenda to reflect detailed requirements for the remaining Round 4 components. This Addendum is the first such communication to the Finalists. B. REPRISE OF ROUND 4 EVALUATION OVERVIEW

Round 4 will include multiple evaluation components, some of which will generate scored assessment by tallied point totals, and others that will serve to draw relative comparisons between the two Finalists in the context of appropriate industry best practices and standards. Concerning the latter form of assessment, the Evaluation Committee may convert comparisons to points on the basis of pre-determined point allocations; i.e., the more favorable comparison results in a larger share of any available points. The entire Round 4 process will facilitate State’s ultimate determination as to which Finalist presents the actual best fit for Contract Award based upon a combination of point totals and more favorable comparisons. Please refer to Attachment 1 to the Addendum for the current Round 4 Maximum Scores by scored component (including Round 4 components addressed in Addendum #11 and this Addendum). By this reference, State incorporates Attachment 1 into the Addendum.

In Round 4, State will measure the following components (not necessarily in the listed order), and may add more as appropriate in reaction to evaluation results:

1. Site Visits to States where Finalists have Solutions in Operation (“Site Visits”);


a. Detailed in Addendum #11 and in process.

2. Maturity Assessment;


3. Contract Redlines Assessment;


4. Revised Price Proposal;


b. In Addendum #11, State previewed a requirement for submission of a high-level approach, plan, schedule and related cost to incorporate Paid Family Medical Leave Insurance (“PFMLI”) tax components into the proposed Solution moved to a separate component.

5. PFMLI Planning and Cost Submission;

a. State requires submission of a high-level approach, plan, schedule and related cost to incorporate Paid Family Medical Leave Insurance (“PFMLI”) tax components into the proposed Solution. State discusses the detailed requirements related to such submission at Section D. of the Addendum.

6. Fiscal Stability of Finalists;

a. Finalists must resubmit original Fiscal Stability materials supplemented with information and data for the following additional period: through 2019.

b. State will not evaluate Fiscal Stability submissions for points. Assessment of the submitted information is for the purpose of validating Contractor Responsibility in the context of their having the capability and capacity to carry out all contractual responsibilities with appropriate financial resources.


a. Periodically detailed in subsequent Addenda as State deems necessary and appropriate.

b. There is one Finalist Follow-up activity detailed in this Addendum at Section G.

B. MATURITY ASSESSMENT SCORING

In Addendum #11, State required Finalists to cooperate with State’s conduct of “Maturity Assessments” through its independent Quality Management Services Consultant. These Maturity Assessments call for independent examination of particular aspects of the Finalists’ UI Operations at designated Travel Sites. The Consultant will conduct these independent examinations in the context of detailed assessment checklists drawn from relevant criteria within the appropriate standard established by the International Organization for Standardization (“ISO”); an international standard-setting body composed of representatives from various organizations promoting worldwide proprietary, industrial, and commercial standards. The appropriate standard for the desired Maturity Assessments is the newest version of the international standard related to assessing software lifecycle processes, ISO/IEC/IEEE 12207:2017, Systems and Software Engineering – Software Life Cycle Processes (“ISO 12207”).

State will receive an assessment report for each Finalist. The report will depict the level of a Finalist’s compliance or non-compliance, with each checklist criterion; and suggested weighting of the ISO 12207 and related criteria based on Contractor’s knowledge and on industry best practices. State will consider such suggestions for its informational purposes, only – not as Contractor’s recommended scoring of Finalists for Contract award.

State will use the results of the Maturity Assessments for the primary purpose of validating a Finalist’s capability and capacity to carry out all contractual responsibilities. Given the qualifications and expertise thus far demonstrated by each Finalist during all rounds of evaluation to this point, State


reasonably assumes that the assessments will suggest that each Finalist has acceptable levels of capability and capacity to successfully perform all contractual obligations.

Examined from State’s perspective, if the results of these assessments support such assumption, and each Finalist has comparable, acceptable levels of capability and capacity to successfully perform; then each will receive 1,000 points. If the results suggest that each Finalist has acceptable levels of capability and capacity to perform, but one Finalist appears more capable in the context of the State-weighted criteria; then the Finalist perceived as more capable will receive 1,000 points and its competitor will receive 750 Points.

C. REVISED COST PROPOSAL SUBMISSION (Due by 5:00 P.M. Pacific Time on April 24, 2020)

State has reviewed the Finalists’ initial Price Proposal submissions, but has not evaluated them for a score. The manner in which the required information is depicted in each, although deemed Responsive in the context of individual submission, defies reasonable comparison between the two submissions for the purpose of scored evaluation. Accordingly, State requires each Finalist to submit revised Price Proposals using the original RFP Attachment E, Price Proposal Form; but with the following clarifying instructions:

1. In preparing their Revised Price Proposals, Finalists should review RFP Attachment A-2, Scope of Services, at Section 12, Maintenance, Support, and Service Levels, and Section 13, Hosting. Take care to ensure that the Revised Price Proposal is responsive to the following instruction from Section 12 of the Attachment: “At a minimum, the Proposal must include an option in which the Proposer is fully responsible for all aspects of maintenance and support. The Pricing Proposal should include pricing for support for at least five years after the end of the stabilization or warranty period.” (Bold, underlined emphasis added). For purposes of the Revised Price Proposal, this instruction also applies to the proposed price for Hosting.

2. Proposed prices for Solution Maintenance and Support and Hosting must at least cover a five-year period after the end of the stabilization or warranty period, whichever is later. Proposed pricing for such period must include all costs for hardware, maintenance, support, and hosting beginning from Solution Go-live (phase-by-phase) and continue through a five-year period from the end of the final stabilization or warranty period, whichever is later. Costs incurred before such time are included as part of implementation costs by phase.

3. OED continues to welcome from Proposers a range of options for support, to allow for the transition of support functions between Contractor resources and internal resources over time.

4. Finally, the Finalists must include with their Revised Price Proposals an option to renew maintenance for an additional five-year period. Finalists may either propose a specific cost option, or link such option to a well-established government price index.

D. PFMLI PLANNING AND COST SUBMISSION (Due by 5:00 P.M. Pacific Time on April 24, 2020)

Finalists must submit a high-level approach, plan, schedule and related cost to incorporate Paid Family Medical Leave Insurance (“PFMLI”) tax components into the proposed Solution. The required submission must be in the context of ORS Chapter 657B, Family and Medical Leave Insurance, and consider the following comparative analysis between UI tax and PFMLI tax. Finalists may exercise their discretion in determining the appropriate format for submission of the required PFMLI approach, plan and schedule; and must submit the related costs in a format comparable to RFP Attachment E, Price Proposal Form. State will not score the initial PFMLI submission for points, but may subsequently create a PFMLI-related Finalist Follow-up activity that it will score for points.

Background. During its 2019 legislative session, the State Legislature decreed that the Director of State’s Employment Department must establish a family and medical leave insurance program to provide family and medical leave insurance benefits to covered individuals as specified by law (ORS 657B.340), and further ordered the Director to establish an advisory committee for the PFMLI program (ORS 657B.380). In the context of this legislation, Finalists shall submit a high-level approach, plan,


schedule, and related cost proposal to incorporate PFMLI tax components into the proposed Solution. State will reserve a contractual option, with no guarantee to its UI Modernization Contractor, to require the Contractor to implement PFMLI tax components into its proposed Solution.

Under the new program, employers are to begin making and remitting contributions into the PFMLI Fund beginning January 1, 2022. (Benefits to eligible employees become payable beginning January 1, 2023.)

Comparative Analysis between UI Tax and PFMLI Tax Programs. There is a large amount of overlap in policy and high-level processes between UI tax functions and PFMLI contributions (tax). Both programs utilize a payroll-based tax to fund their respective trust funds, rely on the same definition of wages to calculate monetary eligibility, and calculate and pay benefits to individuals that have ceased working (UI) or are taking leave for eligible purposes (PFMLI), either partially or totally. In addition, both serve similar population groups, although there are some differences in subjectivity and the ability for populations not generally covered by UI to be covered by, or opt to be covered by, PFMLI. Therefore, the Employment Department anticipates that leveraging a majority of UI Tax functionality and requirements may be possible by incorporating PFMLI into the UI Tax Modernized system.

Both programs can leverage common business functionality, which includes, without limitation; correspondence, employer accounts, employer registration, and payroll reporting. However, there are key differences in the programs as follows:

1. Registration and determination. Key differences from UI tax:

a. Some employers may be subject to PFMLI but not UI.

i. Self-employed individuals and tribal governments may opt in for PFMLI coverage.

ii. Employers that offer an equivalent paid family and medical leave program may apply to opt out of PFMLI contributions.

iii. Employers’ subjectivity to PFMLI may change over time if they begin (or end) offering an Equivalent Plan.

2. Accounting. Key differences from UI tax:

a. Currently, PFMLI contributions (tax) are capped at 1% of wages paid.

b. PFMLI contributions (tax) are paid partially by employers (40%) and partially by employees (60%), with the employer withholding the employees’ contributions from the employee’s pay. PFMLI leverages a high wage cap different than UI’s taxable wage cap.

c. Small employers (fewer than 25 employees) can opt whether to pay their 40% share of the contributions, but are still required to withhold and remit their employees’ 60% contributions.

d. PFMLI is to be self-funded, with the PFMLI Fund established in the State Treasury. A General Fund loan was granted to the department to implement the program, but future costs will be paid out of program contributions.

e. Employment Department staff work performed on PFMLI activities must not bill against UI funds. The system must identify PFMLI activities as non-UI trust fund activities in order to ensure PFMLI and UI funds are billed correctly.

3. Payroll reporting. Key differences from UI tax:

a. Several new data elements would be required to support PFMLI functionality.


b. Self-employed individuals can opt in for PFMLI and would be reporting earnings, generally based on income rather than on traditional payroll data. These individuals would not necessarily be subject to UI Tax, but still need access into the system to report and remit.

c. Small employers may be eligible for grants; requires tracking whether they elect to pay their share of contributions, and tracking which employees work for that employer.

4. Audit. Key differences from UI tax:

a. New data elements may need to be tracked to support PFMLI audits.

5. Reporting. Key differences from UI tax:

a. No federally required PFMLI reports required at this time.

Finalists may access more information about the PFMLI opportunity at:

1. https://www.oregon.gov/employ/PFMLI/Pages/default.aspx

2. https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html

E. REVISED FISCAL STABILITY SUBMISSION (Due by 5:00 P.M. Pacific Time on April 24, 2020)

1. Finalists must resubmit original Fiscal Stability materials supplemented with information and data for the following additional period: through 2019.

2. State will not evaluate Fiscal Stability submissions for points. State will assess the submitted information to validate Contractor Responsibility in the context of their having the capability and capacity to carry out all contractual responsibilities with appropriate financial resources.

F. CLIENT REFERENCE CHECKS

Each Finalist included with its Proposal submissions a list of client references for whom the Finalist implemented and operates or will soon operate completed Solutions comparable to the Solution State seeks to acquire in the current procurement. In RFP Section 3.4.2, State declared that in the Final Procurement Phase it would conduct “Site Visits” for certain Finalist client references. Such visits are currently in progress as “virtual” Site Visits in compliance with the process and procedures described in Addendum #11.

State will check all other listed client references, i.e., those not involved in Site Visits, for each Finalist in the following manner:

1. Provide each client reference a status on State’s underlying procurement;

2. Briefly explain State’s Site Visit process and the seven categories of inquiry related to such visits;

3. Provide a copy of the client reference form (“Initial Client Reference”) submitted on their behalf by a particular Finalist;

4. Request validation of the content reflected in such forms; and

5. Invite a rating from 1 to 5 (very low quality to extremely high quality) for the affected Finalist in the context of their Initial Client Reference and the seven categories of Site Visit queries; encouraging them to provide additional narrative as they deem appropriate.

As indicated in Addendum #11, Site Visits are broad, deep extensions of the client reference check process, which focus on two client references for each Finalist. State will score these Site Visits according to the process described in Addendum #11. Scored Site Visits establish a representative client reference rating for each Finalist. State will use the additional client reference checks to validate or adjust the representative rating upward or downward as previously indicated in Addendum #11.


https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html


G. FINALIST FOLLOW-UP (COVID-19 Measures Due by 5:00 P.M. Pacific Time on April 24, 2020)

Finalist Follow-up activities include possible discussions with, and presentations from, each Finalist to address relevant matters, from State’s perspective, discovered through the course of other Round 4 activity; and in the context of Contractor Responsibility.

State will not evaluate every Finalist Follow-up submission for points, but reserves the opportunity for direct or indirect scored evaluation depending upon the Finalist Follow-up activity. A current example of indirect scored evaluation of Finalist Follow-up activity relates to Client Reference Checks and Virtual Site Visits as related in Addendum #11.

Through this Addendum State imposes its first Finalist Follow-up requirement – COVID-19 Global Pandemic Measures for UI Tax and Benefits programs. The declared COVID-19 Global Pandemic has the world in a tight grip. Human life and economic security are at peril. Countries throughout the world, including the United States of America, are currently following social and economic policies that restrict freedom of personal interaction and movement, constrain economic growth, and impose tremendous burden on government organizations at all levels.

In the context of UI in America, organizations responsible for administering UI tax and benefit programs, like OED, are under pressure to give a helping hand to millions of UI-eligible residents adversely affected by the nation’s response to the Pandemic. Naturally, this circumstance also affects the businesses supporting these government organizations – businesses like Deloitte and FAST Enterprises.

Accordingly, State requires the Finalists to submit documentation (format subject to Finalist discretion) concerning COVID-19 Global Pandemic Measures that Finalists are deploying, or can and plan to deploy, in the following circumstances:

1. Continue effective internal business operations, safely continue to deploy personnel, and effectively market for new business opportunities;

2. Effectively support current customer organizations in compliance with all contractual requirements, and in consideration of the following:

a. In the context of pending Virtual Site Visits and other client reference checking activity, OED will conduct a very deep dive into each affected state’s experience with its UI Contractor concerning operation of and support for their respective UI systems through the rigorous course of the prevailing COVID-19 Pandemic, and the responses thereto required from each state.

b. Of particular interest to OED in the conduct of such inquiry will be discovery of each state’s level of satisfaction with the services of its UI Contractor during this challenging period. How responsive has its UI Contractor been to expressed need? How fast and accurate has its UI Contractor been in making necessary system changes – and at what cost in dollars, risk, downtime and deferred maintenance? These are examples of the depth and breadth of our planned inquiry with each affected state.

c. The nature of the responses to this inquiry from the affected states will be very significant to OED in finalizing scoring related to Site Visits, and in determining the nature of additional Finalist Follow-up activity.

3. One of the Finalists will very likely serve in the near future as OED’s UI Modernization Contractor. How will the Pandemic affect the manner and means of commencement of Services for OED? What immediate assistance will be available to OED as it plans and executes its obligations and opportunities related to the S. 3548 CARES Act?

H. CONCLUSION

Nicholas Betsacon remains as the assigned SPC for this procurement.

Nicholas Betsacon, IT Procurement Strategist




1225 Ferry St. SE.

Salem, OR 97301-4285

Phone: (503) 267-6636


-END OF DOCUMENT (Except for Addendum Attachment 1, which follows)-

mailto:[email protected]



ATTACHMENT 1

ROUND 4 MAXIMUM SCORES BY COMPONENT

No. SCORED COMPONENT POINTS

1

SITE VISITS

Two Site Visits per Finalist. Maximum Score for each Site Visit is 5,000 Points. Total Score is the sum of both scored Site Visits.

10,000

2 CONTRACT REDLINES 300

3


For comparable, satisfactory assessments, each Finalist will receive 1,000 Points. If the assessment reports establish one Finalist as more capable in the context of the State-weighted criteria; then the Finalist perceived as more capable will receive 1,000 points and its competitor will receive 750 Points.

1,000 / 750

4 REVISED COST PROPOSALS 1,500

5 PFMLI SUBMISSION -0-

6 FISCAL STABILITY -0-

7 ADDITIONAL CLIENT REFERENCE CHECKS

Results from these checks do not add “new” points to the evaluation, but may support upward or downward adjustments to Site Visit scores.

-0-

8 COVID-19 PANDEMIC MEASURES -0-

TOTAL 12,800


CSEAS 2.0 PMC Evaluation COVER

Value

Ceiling

(% of Total

Points)

1 0

2 1 – 49%

3 50 – 60%

4 61 – 89%

5 90 – 100%

MAXIMUM

POINTS

5000

5000

0

300

1000

1500

0

0

0

12800

ROUND 4 EVALUATION

TWO FINALISTS | DELOITTE & FAST

DASPS-2232-19 PROCUREMENT EVALUATION

COMMITTEE SCORING SHEET

RATING SCALE

Level 1 –

5*Explanation

RESPONSE OF NO VALUE – Response is unacceptable and does not meet the minimum requirements.

POOR TO FAIR – POOR TO FAIR – Response (including conduct, demeanor, and verbal and written contributions from participating Proposer personnel) meets submittal

requirements but does not demonstrate satisfactory knowledge of the subject matter, or does not demonstrate an ability to deliver requested service.

SATISFACTORY– Response (including conduct, demeanor, and verbal and written contributions from participating Proposer personnel) meets submittal requirements,

demonstrates basic knowledge of the subject matter, and demonstrates an ability to minimally deliver the requested service.

GOOD TO VERY GOOD – GOOD TO VERY GOOD – Response (including conduct, demeanor, and verbal and written contributions from participating Proposer personnel)

meets submittal requirements, demonstrates thorough knowledge of the subject matter, and demonstrates an ability to successfully deliver the requested service.

OUTSTANDING - Response (including conduct, demeanor, and verbal and written contributions from participating Proposer personnel) meets submittal requirements,

demonstrates exceptional knowledge of the subject matter, and demonstrates excellent ability to deliver the requested service.

*5 being the best and 1 being the worst

Evaluation Committee certifies that it has collaboratively

reviewed each Round 4 submission, and collaboratively

scored the selected Round 4 submissions, as designated in

this spreadsheet in full compliance with the screening,

selection, and evaluation criteria set out in RFP DASPS-

2232-19; and that the scores reflected are accurate and

final. Committee has submitted any written comments

regarding any of the designated proposals via seperate

documentation to the Single Point of Contract for this

procurement.

VIRTUAL SITE VISIT 1


CLIENT REFERENCE CHECKSNo separate scoring, but results may impact Virtual Site

Visit scoring as addressed in Addenda 11 and 12

SIGNATURE

BOX

FINALISTS:

DELOITTE

FAST ENTERPRISES

CONTRACT REDLINES



REVISED COST PROPOSAL

REVISED FISCAL STABILITY

PFMLI TAX SUBMISSIONS

COVID-19 PANDEMIC SUBMISSIONS

Document Number: 5228580

CSEAS 2.0 PMC Evaluation ROUND 4 FINALISTS

Value

Ceiling

(% of Total

Points)

1 0

2 1 – 49%

3 50 – 60%

4 61 – 89%

5 90 – 100%

PER-POINT

FACTOR

Insert Rating

Number

Calculated Points:

Per-Point Factor

multiplied by Rating

Number

Insert Rating

Number

Calculated Points:

Per-Point Factor

multiplied by Rating

Number

5000 3205 4340

Vendor Relationship

500100 2.60 260 4.00 400

Implementation

1250250 3.80 950 4.20 1050

Maintenance & Support

1250250 2.50 625 4.80 1200

Contract

25050 3.80 190 4.00 200

Solution Overview

1000200 3.40 680 4.20 840

End User Experience

500100 2.50 250 4.00 400

Lessons Learned

25050 5.00 250 5.00 250

5000 3205 4340

Vendor Relationship

500100 2.60 260 4.00 400

Implementation

1250250 3.80 950 4.20 1050

Maintenance & Support

1250250 2.50 625 4.80 1200

Contract

25050 3.80 190 4.00 200

Solution Overview

1000200 3.40 680 4.20 840

End User Experience

500100 2.50 250 4.00 400

Lessons Learned

25050 5.00 250 5.00 250

0

SITE VISITS

1-2

6410

SITE VISITS

1-2

8680

300 300 300

1000 1000 1000

1500 1421.2 1500

0

0

0

12800 9131.2 11480

CONTRACT REDLINES

MATURITY ASSESSMENT


REVISED FISCAL STABILITY

PFMLI TAX SUBMISSION

COVID-19 PANDEMIC SUBMISSION

REVISED COST PROPOSAL

RATING SCALE

Level 1 –

5*Explanation

*5 being the best and 1 being the worst

DASPS-2232-19 PROCUREMENT EVALUATION

COMMITTEE SCORING SHEET

CLIENT REFERENCE CHECKSNo separate scoring, but results may impact Virtual Site Visit

scoring as addressed in Addenda 11 and 12

DELOITTE

RESPONSE OF NO VALUE – Response is unacceptable and does not meet the minimum requirements.

POOR TO FAIR – POOR TO FAIR – Response (including conduct, demeanor, and verbal and written contributions from participating Proposer

personnel) meets submittal requirements but does not demonstrate satisfactory knowledge of the subject matter, or does not demonstrate an ability

to deliver requested service.

SATISFACTORY– Response (including conduct, demeanor, and verbal and written contributions from participating Proposer personnel) meets

submittal requirements, demonstrates basic knowledge of the subject matter, and demonstrates an ability to minimally deliver the requested service.

GOOD TO VERY GOOD – GOOD TO VERY GOOD – Response (including conduct, demeanor, and verbal and written contributions from participating

Proposer personnel) meets submittal requirements, demonstrates thorough knowledge of the subject matter, and demonstrates an ability to

successfully deliver the requested service.

OUTSTANDING - Response (including conduct, demeanor, and verbal and written contributions from participating Proposer personnel) meets

submittal requirements, demonstrates exceptional knowledge of the subject matter, and demonstrates excellent ability to deliver the requested

service.

FAST ENTERPRISES



ROUND 4 EVALUATION

TWO FINALISTS | DELOITTE & FAST


CSEAS 2.0 PMC Evaluation SITE VISITSDELOITTE

Massachusetts 1 2 3 4 5 6 7 AVG ADJ-AVG

Vendor Relationship 1 1 1 1 1 2 1 1.1 2.6

Implementation 2 2 3 2 2 3 2 2.3 3.8

Maintenance&Support 1 1 1 1 1 1 1 2.5

Contract 3 3 2 2 1 2 3 2.3 3.8

Solution Overview 2 2 2 2 1 2 2 1.9 3.4

End User Experience 1 1 1 1 1 1 1 1 2.5

Lessons Learned 5 5

FAST

Washington 1 2 3 4 5 6 7 AVG ADJ-AVG

Vendor Relationship 4 4 4 4 4 4 4

Implementation 4 4 5 4 4 4.2 4.2

Maintenance&Support 5 4 5 5 5 4.8 4.8

Contract 4 4 4 4 4 4 4

Solution Overview 4 4 4 5 4 4.2 4.2

End User Experience 4 4 4 4 4 4 4

Lessons Learned 5 5

DUE TO IMPACT OF COVID-19 PANDEMIC RESPONSE OBLIGATIONS ON STATES VISITED, THE SITE VISITS

WERE LIMITED TO 1 FOR EACH FINALIST. PROFILE SCORE DOUBLED TO FIT WITHIN THE 10,000 POINT

PROFILE. ADJUSTMENTS AS APPROPRIATE BASED ON OTHER CLIENT REFERENCE RATINGS.

VALUABLE LESSONS LEARNED IN EACH SITE VISIT FOR EACH FINALIST. REGARDLESS OF IMPACT ON

PERCEIVED FINALIST PERFORMANCE (NEGATIVE OR POSITIVE) EACH FINALIST IS AWARDED MAX POINTS IN

THE LESSONS LEARNED COMPONENT.

Client Reference

Factor Added

1.5

Client Reference

Factor Added

0
