kathy downey, dee mccarthy, william mccarthy, and brian meekins u.s. bureau of labor statistics

1

Encouraging the Use of Alternative Modes of Electronic Data Collection: Results of Two Field Studies

Kathy Downey, Dee McCarthy, William McCarthy, and Brian Meekins

U.S. Bureau of Labor Statistics

The opinions expressed here are those of the authors and do not represent official policy of the Bureau of Labor Statistics.

2

BLS Internet Data Collection

First begun in the 2002 survey year Termed the “IDCF” – Internet Data

Collection Facility Uniform and secure structure for

employers to submit data for BLS surveys

Designed to look very similar to the Survey of Occupational Injuries and Illnesses data collection booklet

Savings in mail costs, print costs, data entry costs

3

Survey of Occupational Injuries and Illnesses

Federal/State cooperative survey Sample of 230,000 establishments per year

(private, State and local gov’t.) Data collected based on OSHA logs –

information on illness/injury: Total recordable cases, Cases with days away from work (with more

about demographics and case characteristics), Cases with days of job transfer or restriction

Hardcopy is 12 pages, envelope and insert Web form looks similar to hardcopy form

4

Internet Data Collection: Number of Days-Away-From-Work Cases Submitted

10,595

23,36929,157

51,882

29,551

50,707

0

10,000

20,000

30,000

40,000

50,000

60,000

2002 2003 2004

Estabs Cases

5

Purpose of Tests

Expanding use of IDCF to Reduce processing time Capture narrative text of days-

away-from-work cases Lower printing and postage costs

6

Tests 2005 SOII: January – June 2006

1st mailing: January 2006 1st non-response mailing: Feb 14 2nd non-response mailing: April 5 Handed over to States for calls (end test): May 9 Close-out: June 26

2006 SOII: January – July 2007 1st mailing: January 4 2007 1st non-response mailing: March 2 2nd non-response mailing: April 20 Handed over to States for calls (end test): TBD Close-out: July 17

7

2005 Test: Sample

2,000 units in each of three tests and a control group

6,688 units that used IDCF in 2004 and were in 2005 sample – termed INET units and received the Test 2 booklet

8

2005 Test: Booklets Test 1

4 page booklet with electronic options* and a phone number to request standard booklet – all 3 mailings

Test 2 plus INET 4 page booklet with electronic options* and a phone

number for help – all 3 mailings Test 3

4 page booklet with electronic options and a phone number for help – received standard booklet on second nonresponse mailing

Control group standard booklet

*use IDCF or send an e-mail to get an electronic copy of the survey

9

10

11

Analyses

Performance Response rate

# resp / “viables” (excludes OOB, OOS, duplicates)

Internet response rate # internet resp / “viables”

Burden Percentage of units contacting Help

phone number Added processing time and costs

12

2005 Internet Data Collection: Establishments and Days-Away-From-Work Cases

10,595

23,369

29,157

51,882

29,551

50,70753,575

70,783

010,00020,00030,00040,00050,00060,00070,00080,000

2002 2003 2004 2005

Estabs Cases

13

Survey Year 2005 Tests Results(at end of test – May 9, 2006)

71.1

46.1

71.3

49.5

73.5

47.2

90.3

76.4 78.4

21.7

80.6

25.0

0102030405060708090

100

Test 1 Test 2 Test 3 INET ControlGroup

Allunits

Response rate IDCF response rate

14

87.1

49.2

86.6

53.4

89.8

49.6

96.1

78.5

91.6

23.1

91

25.8

0102030405060708090

100

Test 1 Test 2 Test 3 INET ControlGroup

Allunits

Response rate IDCF response rate

Survey Year 2005 Tests Results(Final)

15

Survey Year 2005 Tests Results(at end of test – May 9, 2006)

14.1

11.7

9.8

5.0

6.4

3.6

10.3

0.6

0

5

10

15

Test 1 Test 2 Test 3 INET

% contacting % wanting booklet

16

Why 2006 Test?

Expand test to larger sample (88,421 units vs. 12,688 units) on a State-level basis

Use States to more fully resemble national implementation Eliminate calls to National Office

(switch to State for help) Estimate data entry burden for States

Improve how respondents access IDCF (move to permanent IDs)

17

2006 Test Test Sample

16 States 8 State partners: CA, DC, FL, IL, MD, NC, NY, VA 8 States by Reg. office: CO, ID, MS, ND, NH, OH,

PA, SD 13,297 INET units Total: 88,421 units received the electronic

options booklet Sampling was modified in the 2006 survey

year to allocate more sample to establishments predicted to have cases

Use only test 2 scenario - 4 page booklet with electronic options* and a phone number for help – all 3 mailings

18

2006 Internet Data Collection: Establishments and Days-Away-From-Work Cases

10,59523,369 29,157

51,882

29,551

50,70753,575

70,783

82,445

157,705

85,974

164,455

0

40,000

80,000

120,000

160,000

200,000

2002

2003

2004

2005

2006

(thru

5/29

)

2006

projected

Estabs Cases

19

74.8

52.4

89.8

77.783.4

28.1

80.6

36

0

10

20

30

40

50

60

70

80

90

Test INET Rest All units

Response rate IDCF response rate

Survey Year 2006 Results(through April 15, 2007)

20

44.4

51.955.655.555

69.4 7378.379.380.8

19.7

25.532.6

3637.427.3

33.140.2

43.945.7

0102030405060708090

Test INET Rest All units

1-10 emp 11-49 emp 50-249 emp250-999 emp 1000+ emp

Survey Year 2006 Results(through April 15, 2007)

21

52.4

8.27.3

3.8 3.1

77.7

5.84.2

0.9 1.2

28.1

51.9

1.1 1.3 1

3637.7

3.1 2.1 1.70

10

20

30

40

50

60

70

80

Test INET Rest All units

IDCF Mail E-mail Telephone FAX

Survey Year 2006 Results(through April 15, 2007)

22

Going back to Purpose of Test…

Reduce processing time Capture narrative text of days-

away-from-work cases Lower printing and postage costs

23

3

6 6

9

13

0

10

20

Phone Mail(Contractor)

Fax E-mail Mail (State)

2006 Test: Lowered Processing Time (IDCF benchmark)

24

6%

32% 32%36%

95% 98%

61%

0%10%20%30%40%50%60%70%80%90%

100%

Mail (State)E-mail Fax Phone Mail(contractor)

IDCF All

2006 Test: Captured More Case Narratives (Percent as of May 3)

25

2006 Test: Lowered Cost (IDCF benchmark)

Per Year - If assumed all cases submitted by IDCF, and sample of 230,000

Booklet Approx mail cost

Approx print cost

Approx total cost

12-page $382,191 $262,808 $644,999

4-page $171,327 $216,142 $387,469

Difference $210,864 $46,666 $257,530

26

Another Option: E-mail Data Collection

First begun in the 2004 survey year Microsoft Word template received from e-

mail request Filled in template transmitted to State

specific e-mail address that routes the form through BLS and to the correct State agency

27

Another Option: E-mail Data Collection

914

2,475

8,809 9,122

0

2,000

4,000

6,000

8,000

10,000

2004 2005 2006 (thru5-29)

2006(projected)

Estabs

28

Conclusions Was successful

53% used IDCF when offered, another 8.5% use E-mail

INET group getting bigger Capturing more case narratives

Explore why not getting higher response rate overall (8 pts) Will run another study of 16 States,

plus 4 more Change instruction booklet to maybe

look less like survey