Summative Evaluation of Soundprint’s Pole to Pole (NSF #0632194): An Exploratory Study of the Impact of Radio Documentaries on Listener Understanding of Science Research on Climate Change

Summative Evaluation of Soundprint's Pole to Pole (NSF #0632194): An Exploratory Study of the Impact of Radio Documentaries non Listener Understanding of Science Research on Climate Change (This report was published in The Informal Learning Review, #103, July/August 2010.) Robert L. Russell, Learning Experience Design Washington, DC eldrbob@gmail.com Soundprint Media was awarded a grant from the National Science Foundation in 2007 to produce eight half-hour radio documentaries on scientific research in the Polar Regions. The programs have been broadcast on participating National Public Radio affiliates. Two programs that focused on climate change research were selected as the focus of the summative evaluation study. This evaluation study was designed with two broad objectives. The first was to gather objective data on what listeners heard and retained from the two programs by having participants fill out a written survey. The second was to gather self-report data on how the programs affected participants' views of science research and climate change. Two focus groups discussions were conducted in Washington, DC, in May 2010; each lasted approximately one hour. Focus group participants were recruited through notices on neighborhood listserves and Craigslist in the Washington, DC area. A total of 20 participants were recruited for the two focus groups sessions, which were held in Washington, D.C. The composition of the focus group groups paralleled NPR listener demographics. Thus, the group was older, reported higher incomes, and had higher levels of education than national averages. Participants listened to the two 30-minute programs prior to attending the session. Once at the session, participants were asked to fill out a survey asking for demographic data (age, race, education level), radio listening habits, and their recall of the contents of the programs. After filling out the survey, the participants filled out a survey asking about their recall of program contents. After the surveys, participants took part in a focus group session that lasted about one hour. The results can be briefly summarized as follows for the written survey (#1-3 below) and focus group discussions (#4-8 below): 1. Most participants accurately described one or more big ideas or themes presented in the programs. In the written survey, 90% of participants accurately reported at least one big idea or theme for Climate Change College with 80% doing the same for When Snow Melts in Svalbard. 2. Most participants accurately listed one or more research topics or techniques. In the written survey, 90% of participants were able to list at least one research topic or technique for Climate Change College; with 75% doing the same for When Snow Melts in Svalbard. 3. A majority of participants reported learning at least one concept from the programs. In the written survey, 90% of participants reported learning at least one new concept from Climate Change College with 60% reporting the same for When Snow Melts in Svalbard. These lower percentages may be partially explained by two factors. First, in the focus group discussion, some participants reported that much of the content was familiar to them. Second, in the focus group discussions, some participants reported that the content of When Snow Melts in Svalbard was more technical and more difficult to understand than the other program. 4. The programs had a positive impact on two-third of participants. A large majority of participants reported that the programs had increased their interest in climate change, changed their thinking, motivated them to want to learn more, or encouraged them to advocate for actions to address the issue. (Two-thirds also are reported that, prior to the program, they accepted that climate change was occurring.) 5. Participants were very engaged by the impact of climate change on another culture. Some participants reported they when they heard news on climate change, it seemed abstract and not linked to impacts on specific communities. They reported that being very interested and engaged in how climate change was having significant impacts on the Inuit. 6. Scientists in the programs were seen as dedicated and open-minded professionals pursuing their research in difficult environments. The programs were effective in introducing scientists as real people who worked hard, worked outdoors, and had active intellects. This contrasted the stereotype some participants reported they had about scientists as bookworms confined to their labs. 7. Participants seemed surprised by the nitty gritty and laborious nature of research. Most of the participants seemed unfamiliar with what scientists actually do on a daily basis, with some reporting that when they think of science research, they think of scientists in labs. The programs effectively conveyed some of the research techniques and nuts and bolts aspects of climate change research. 8. Participants believed the research was conducted through a process of inquiry. They mentioned examples of scientists in the programs who described how they had changed their minds. The programs were effective in presenting science as a process of asking questions and gathering data to get the answers. In summary, while participants generally supported the view that climate change results from human activity, they did not have an in-depth understanding of climate change research prior to the programs. The programs were effective in introducing the nitty gritty nature of the research and the scientists' approach to research - that it is a process of inquiry and not just seeking data that supports prior views. A great majority of participants were able to recall big ideas and specific research topics and techniques introduced in the programs. In brief, the Soundprint Media programs resulted in significant cognitive impacts and were successful in deepening the understanding that participants had of climate change research, climate change researchers, and the impacts climate change is having on humans and the environment.These results suggest that media producers and informal science educators should place a greater emphasis on engaging the public in learning about who conducts science research and how it is conducted. Participants were often surprised when learning about the research techniques and finding out what the scientists in the programs were like as real people. The results also suggest that showing the impact of climate change on people and their local environments is effective in engaging interest in the topic.

If you would like to edit a resource, please email us to submit your request.