federal consulting group 2004 customer satisfaction study may 2004 nasa earth observing system data...
TRANSCRIPT
Federal Consulting Group
2004 Customer Satisfaction Study
May 2004
NASA Earth Observing System Data and Information Systems
2
Federal Consulting Group
Snapshot of the American Customer Satisfaction Index (ACSI)
The #1 national economic indicator of customer satisfaction
Compiled by the National Quality Research Centerat University of Michigan Since 1994 usingmethodology licensed from CFI Group
Measures 40 industries, 200 organizations covering 75% of the U.S. economy
– Over 70 U.S. Federal Government agencies have used ACSI to measure more than 120 programs/services
Advanced methodology quantifiably measures and links satisfaction levels to performance and prioritizes actions for improvement
3
Federal Consulting Group
Why Should Agencies Measure Customer Satisfaction?
Link customer satisfaction with expectations and
desired outcomes
Benchmark against “best” in business and
government
Set “baseline” for customer satisfaction and measure progress
Provide critical information for annual performance plans to Congress (as required
under GPRA)
Identify areas for improving quality of service provided
to customers
Raise trust in your agency and the
government overallEnable Senior Executive
Service members to meet performance criteria
Customer Satisfaction
4
Federal Consulting Group
Review of ACSI Results
A component score is a weighted average of the set of survey questions comprising a component or activity area. Responses to survey questions are given on a 1-10 scale, which is converted to a 0-100 scale for score reporting.
An impact predicts the increase in satisfaction that would result from a 5-point increase in a component or input score.
Areas for improvement are those components or activities with a relatively low score and a relatively high impact on satisfaction.
EXAMPLE
0.8
79
76
65
1.2
Component 2
Component 1
ACSI
Score
Impact
In the simplified example shown here,
Component 2 would be a key action area due
to its relatively low score and high impact.
5
Federal Consulting Group
Which NASA EOSDIS customer segment was surveyed?
This customer segment includes individuals who have accessed NASA EOSDIS data and/or products by means of a NASA EOSDIS Data Center (DAAC).
How were the NASA EOSDIS customers identified?
NASA EOSDIS provided a list of 33,251 email addresses for people who have used NASA EOSDIS data and/or products. A sample drawn from the list, and CFI Group sent out 9,999 email invitations asking customers to participate in the online survey.
The survey was available online April 29, 2004 – May 19, 2004. A total of 1,056 surveys were completed, of which CFI Group used 1,016 surveys for analysis.
Project Background
6
Federal Consulting Group 75*
NASA EOSDIS Results
The Customer Satisfaction Index for NASA EOSDIS is…
*The confidence interval for ACSI is +/-1.1 for the aggregate at the 95% confidence level.
NASA EOSDIS Aggregate Segment
The Customer Satisfaction Index score is derived from customer responses to three questions in the survey:
– How satisfied are you overall with the products and services provided by the Data Center (79)?
– To what extent have the data, products and services provided by the Data Center fallen short of or exceeded your expectations (73)?
– How well does the Data Center compare with an ideal provider of scientific data, products and services (71)?
This score is four points higher than the 2003 American Customer Satisfaction Index for the Federal Government overall (71).
7
Federal Consulting Group
Comparison Across Government Agencies
While NASA EOSDIS performs well when compared to the Federal Government overall, it doesn’t perform as well when compared to other information providing agencies.
Customer Satisfaction Index
76
76
78
79
75
71
74
60 70 80 90 100
HRSA - HIPDB Queryers
NWS - Media
HRSA - NPDB Queryers
GSA - FCIC
NASA EOSDIS
Federal Govt Overall
ACSI Overall
8
Federal Consulting Group
NASA EOSDIS Customer Satisfaction Model
Product Search
70
Delivery84
Product Quality
68
Customer Support
84
Product Selection and Order
73
Customer Satisfaction
(ACSI)75
Future Use90
Recommend86
Customer
Complaints
32%
Quality Components (Drivers)
0.8
1.1
0.9
0.5
1.0
3.5
2.5
0.6
ACSI Outcomes
9
Federal Consulting Group
Summary of Findings
Product Quality is the lowest scoring component (68), and has a relatively high impact (0.9).
– All attributes in this area received similar ratings
At 84 Customer Support scores well, but is also high impact (1.0).– There is a significant difference in Customer Support ratings given by
customers within the U.S. (88) compared to those outside the U.S. (82).
The components Product Search and Product Selection and Order are highly correlated.
Recent customers are more satisfied, but are also reporting more problems.
Percent of Customer Complaints is fairly high (32%) when compared to the Federal Government overall (12%).
– Customers may not be calling to complain about a problem, but rather to seek assistance in solving the problem.
– 90% of respondents who answered the Customer Complaint questions gave user services’ complaint handling a rating of “6” or above.
10
Federal Consulting Group
Where Should NASA EOSDIS Focus?
Maintain Building on Success
Delivery (84/0.5)
Minimum Attention Now
Top Priority
Product Selection and Order (73/1.1)Product Quality (68/0.9)Product Search (70/0.8)
Customer Support (84/1.0)
LOW Impact on Satisfaction (CSI) HIGH
LOW
Sco
re H
IGH
11
Federal Consulting Group
Product QualityImpact: 0.9
68
69
67
68
Product Quality
Ease of using thedata product in the
delivered format
Clarity of dataproduct
documentation
Thoroughness ofdata product
documentation
Was documentation…
Delivered with the data 44%
Pointed to (on a website) 41%
Not available 15%
In what format were data or products provided?
HDF-EOS 49%
HDF 39%
NetCDF 5%
Binary 14%
ASCII 12%
GeoTIFF 19%
Other 7%
12
Federal Consulting Group
Product SearchImpact: 0.8
70
69
68
73
Product Search
Ease offinding/accessing
data
Ease of using searchcapability
How well the searchresults met your
needs
13
Federal Consulting Group
Product Selection and OrderImpact: 1.1
73
70
74
72
74
Product Selectionand Order
Ease of identifyingappropriate data
products
Completeness of dataproducts
Description of dataproducts
Ease of using onlineordering form
14
Federal Consulting Group
Customer SupportImpact: 1.0
84
83
82
84
85
85
87
Customer Support
Professionalism
Technical knowledge
Accuracy of information provided
Helpfulness in selecting/finding data orproducts
Helpfulness in correcting a problem
Timeliness of response
15
Federal Consulting Group
DeliveryImpact: 0.5
84
86
81
Delivery
Convenience ofdelivery method
Timeliness of deliverymethod
66% of customers’ data was delivered through FTP
16
Federal Consulting Group
Have contacted EOSDIS
32%Have not contacted EOSDIS
68%
Desired Outcomes
NASA EOSDIS wants its customer to recommend and use EOSDIS services in the future. Desired customer behaviors were measured with a single question each.
– “How likely are you to recommend the Data Center to a colleague?” scored an 86 (Impact of CSI onto Recommend: 3.5.)
– “How likely are you to use the services provided by the Data Center in the future?” scored a 90 (Impact of CSI onto Future Use: 2.5.)
NASA EOSDIS customers were also asked if they had ever contacted the Data Center’s user services office to report a problem.
– 32% of of customer say they have contacted the Data Center’s user services office to report a problem.
17
Federal Consulting Group
Score ComparisonTime of Last Data Product or Service Request
76
68
37%
72
79
83
70
65
66
23%
86
86
74
72
ACSI
Delivery
CustomerSupport
Product Selectionand Order
Product Search
Product Quality
Complaints
Less than 3months ago(n=723)
Over 3months ago(n=238)
18
Federal Consulting Group
Score ComparisonCurrent Location
74
67
34%
76
82
83
73
71
69
31%
88
85
72
69
ACSI
Customer Support
Delivery
Product Selectionand Order
Product Search
Product Quality
Complaints
USA
Outsidethe USA
10 point difference in the score for “timeliness of
response”
19
Federal Consulting Group
Score ComparisonData Centers
ACSI
LaRC DAAC7%
LP DAAC22%
Other26% GES DAAC
27%
ASF DAAC1%
PO DAAC7%
ORNL DAAC3%
NSIDC DAAC5%
GHRC2%
77
77
74
74
74
71
72
77
71
ASF DAAC
GES DAAC
GHRC
LaRC DAAC
LP DAAC
NSIDC DAAC
ORNL DAAC
PO DAAC
Other
20
Federal Consulting Group
Recommendations
Focus on Product Quality.– Review the type of data product documentation available with
each product. Work to improve the clarity and thoroughness of the documentation.
– Assess the various data formats and work to improve the usability of each.
– Offer a wider variety of data formats.
Review the Product Search and Product Selection and Order scores to determine how best to help customers find the data they need.
– Due to high correlation, improvements in one area will likely result in improvements in the other.
– Simplify the search process; make data products more apparent.– Improve data product descriptions.
21
Federal Consulting Group
Additional Information
Detailed survey results for all of the Federal services, including trends in performance and customer satisfaction, were updated in December 2003 and can be found on the website www.customerservice.gov.
Many agencies share best practices through the interagency customer service forum.