survey says: the latest data on routing

Brief Review of Research on RoutersSeptember 9, 2013

Key Findings from Three Papers

Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.

Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results. Presentation given at the 2012 CASRO Online Conference.

Johnson & Fawson Reference the ”Tragedy of the Commons”

Wikipedia Says

In economics, the tragedy of the commons is the depletion of a shared resource by individuals, acting independently and rationally according to each one's self-interest, despite their understanding that depleting the common resource is contrary to the group's long-term best interests. "Commons" can include the atmosphere, oceans, rivers, fish stocks, national parks, advertising, and even parking meters.

And, even online sample!

Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.

Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.

Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.

Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.

Johnson & Fawson Conclusions & Suggested Controls

Source: Johnson, Paul, & Bob Fawson (2010). Factorial Design on Survey Router Effects. Paper given at the 2010 CASRO Panel Conference.

Brigham et al: Reallocation

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Source: Brigham, Nancy, Scott Porter, Lee Markowitz, & Jason Fuller (2011). Sampling with Routers: Comparing Survey Results of “Reallocated” and Traditionally Sampled Respondents. Presentation given at the 2011 CASRO Technology Conference.

Brigham & Fuller Paper Sought to Answer Three Questions

Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results. Presentation given at the 2012 CASRO Online Conference.

1. Does spending more time in the router screening process have an impact on results?

2. Does correlation among screening questions matter? Should a router be diverse in the types of studies it includes?

3. How small can a router be – are a minimum number of studies needed?

“Spending a greater amount of time in the router was not related to the screening process.”

Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results. Presentation given at the 2012 CASRO Online Conference.

“A highly correlated router produced skewed results; including a mixture of correlations brought results back into line.”

Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results. Presentation given at the 2012 CASRO Online Conference.

“More studies are better – Test metrics are closer to Control around 10-12 surveys in a router.”

Source: Brigham, Nancy, Jason Fuller (2012). Survey Router Management: An Experimental Examination of Impact on Survey Results. Presentation given at the 2012 CASRO Online Conference.

What about FOQ2?

(Chuck?)