Survey with Placebo: A Method for Evaluating the Influence of Advocacy
In December 2013, the Brown Center on Education Policy at the Brookings Institution released “Measuring the Influence of Education Advocacy: The Case of Louisiana’s School Choice Legislation,” a report which introduces a new survey methodology for gauging the influence of policy advocacy efforts. Surveys, as they are commonly used in advocacy evaluation, often ask key stakeholders to assign rankings or scores to specific advocacy organizations based on their perceived level of policy influence. However, this approach assumes that the starting point for these scores is zero, which may not be the case. To address this survey design challenge, the report’s authors developed an instrument they call Survey with Placebo (SwP). SwP also collects ratings from select respondents, but with one important difference: among the advocacy groups respondents are asked to score, there is a “placebo,” a non-existent organization with a plausible name. The scores assigned to this placebo are then used as the zero-point on an influence scale against which all other responses are measured, thereby facilitating the use of statistical tests to quantify the amount of influence of each advocacy organization. Brown Center Director Grover “Russ” Whitehurst and Basis Policy Research Managing Partner David Stuit will present this new method and findings from its initial implementation on the passage of school choice legislation in Louisiana.
Presenters:
Grover “Russ” Whitehurst is the Herman and George R. Brown Chair and director of the Brown Center on Education Policy at the Brookings Institution. Previously, he was director of the Institute of Education Sciences at the U.S. Department of Education; US assistant secretary for Educational Research and Improvement; chair of the Department of Psychology at the State University of New York at Stony Brook; and academic vice-president of the Merrill-Palmer Institute. He received his Ph.D. in experimental child psychology from the University of Illinois at Urbana-Champaign in 1970. His specializations include program evaluation, teacher quality, preschools, national and international student assessments, reading instruction, education technology, and education data systems.
David Stuit is founding and managing partner of Basis Policy Research in Raleigh, North Carolina, where he focuses on delivering quantitative research solutions to Basis clients. David has served as the Director of Business Intelligence for a leading education management organization, a researcher with the National Center on School Choice and the Education Commission of the States and as a classroom teacher in Denver, Colorado. In addition to a Ph.D. from Vanderbilt University, David holds a master’s degree from the University of Colorado and a Bachelor’s from Hope College.