This article addresses some of the challenges faced when attempting to evaluate the long-term impact of informal science learning interventions. To contribute to the methodological development of informal science learning research, we critically examine (Falk and Needham (2011) Journal of Research in Science Teaching, 48: 1–12.) study of the California Science Center's long-term impact on the Los Angeles population's understanding, attitude and interest in science. This study has been put forward as a good model of long-term impact evaluation for other researchers and informal science learning
Access to high quality evaluation results is essential for science communicators to identify negative patterns of audience response and improve outcomes. However, there are many good reasons why robust evaluation linked is not routinely conducted and linked to science communication practice. This essay begins by identifying some of the common challenges that explain this gap between evaluation evidence and practice. Automating evaluation processes through new technologies is then explicated as one solution to these challenges, capable of yielding accurate real-time results that can directly
Many environmental communication interventions are built on the assumption that increased knowledge will lead to changes in proenvironment behaviors. Our study probes the link between biodiversity-related knowledge and self-reported proconservation behavior, based on the largest and most international study of zoo visitors ever conducted. In total, 6,357 visitors to 30 zoos from 19 countries around the globe participated in the study. Biodiversity understanding and knowledge of actions to help protect biodiversity were significantly related, but only 0.6% of the variation in knowledge of
Campaigns by zoos, aquariums, and other civil society organizations are an important tool for promoting social changes that benefit the environment. Here, we evaluate a global biodiversity education campaign's impact through a repeated-measures survey of nearly 5000 visitors to 20 zoos and aquariums located in 14 countries. By comparing visitors’ pre- and post-visit responses combined across respondents, we found significant aggregate improvements in their biodiversity understanding and their knowledge of actions to help protect biodiversity. Respondents who reported seeing the education
Scientific institutions are increasingly embracing values of inclusivity and public engagement, but how do these two dimensions intersect? Science festivals have rapidly expanded in recent years as an outgrowth of these values, aiming to engage and educate the public about scientific topics and research. While resources invested in public engagement by scientists, universities, and governments are admirable in principle; this study indicates that their ambition to broaden the reach of science may be going unrealized in practice. Using data from three major UK science festivals, we demonstrate
Zoos and aquariums aim to achieve lasting impact on their public audiences’ awareness of biodiversity, its value, and the steps they can take to conserve it. Here, we evaluate the long-term educational impact of visits to zoos and aquariums on biodiversity understanding and knowledge of actions to help protect biodiversity. A minimum of two years after completing a repeated-measures survey before and after visiting a zoo or aquarium, the same participants were invited to take part in a follow-up online survey. Despite the small number of respondents (n = 161), our study may still represent the
The Tech Museum of Innovation (The Tech) in San Jose, California proposes to partner with NOAA to integrate Science On a Sphere (SOS) into The Tech's Exploration gallery and to facilitate the development of informal and formal learning programs. Exhibits and programs at The Tech focus on the integration of emerging technologies into hands-on visitor experiences. In 2004, The Tech partnered with NOAA, the Maryland Science Center (MSC), and a consortium of national science centers to explore the potential and effectiveness of SOS as a method of engaging and informing the general public about NOAA-related sciences. Initial testing of SOS at the Maryland Science Center revealed that SOS is a visually compelling and engaging medium for conveying complex scientific information to museum visitors. Ninety-eight percent of visitors tested regarded a facilitated SOS program as a good or excellent experience with strong visitor retention suggesting the potential of SOS as a compelling visitor tool. However, when the experience was not facilitated this retention dropped dramatically. Support from NOAA will enable The Tech to test SOS and NOAA data in a number of formats to determine the most effective ways to utilize this incredible technology. The results of this evaluation will be shared with other museums using SOS to improve its reach in teaching informal audiences and promoting interest in both STEM content and NOAA research. The SOS exhibit will bring together scientists, technologists, informal education specialists, and young users to unlock the educational potential of NOAA's datasets and further NOAA's educational plan. Hands-on experiences using SOS will engage visitors in meaningful explorations of NOAA data. The Tech Museum will make SOS accessible to people of all ages, backgrounds, and educational levels. All panel text, audio, and captions will be presented in both English and Spanish to allow greater accessibility for local audiences. SOS will provide the programming platform upon which to explore the educational opportunities of this gallery as it illustrates how data collected with remote sensing technologies is helping us understand and make predictions about our dynamic environment and the future of our planet. SOS will illustrate how these data collecting technologies assist us in developing our knowledge about our planet and its solar system.
The FIRST Longitudinal Study is a multi-year longitudinal study assessing the impacts of FIRST’s afterschool robotics programs on the STEM related interests and educational and career trajectories of program participants. FIRST is one of the nation’s largest after-school robotics programs, serving more than 460,000 youth aged 6-18 annually through the FIRST LEGO League (Ages 7-14), the FIRST Tech Challenge (grades 7-12) and the FIRST Robotics Competition (grades 9-12). The study is tracking over 1200 program participants and comparison students, using a quasi-experimental design, over a
DATE:
TEAM MEMBERS:
Alan MelchiorCathy BurackMatthew HooverJill Marcus
The Ocean Science project integrates the Ocean Literacy Essential Principles and Fundamental Concepts into a Western Washington region-wide, coordinated program of formal and informal education consisting of: 1. Teacher professional development in the ocean sciences to integrate the Ocean Literacy Essential Principles and Fundamental Concepts into inquiry-based marine science education and instruction; 2. Evaluation and re-alignment of existing Sound Science ecosystems curricula into Ocean Science, incorporating NOAA data and promoting the Ocean Literacy Essential Principles and Fundamental Concepts; 3. Classroom programs, beach field investigations, and on-site programs at the Seattle Aquarium of the Olympic Coast national Marine Sanctuary's Olympic Coast Discovery Center for grades 4-5 students, their parents and teachers; 4. Parent training in ocean science content, the Ocean Literacy Essential Principles and Fundamental Concepts, and inquiry-based methods for supporting their children's science education; 5. Informal education for the general public via an interactive learning station linked to the Window on Washington Waters exhibit and designed to innovatively use NOAA data and information (videos, computer simulations and other creative media) to increase and evaluate ocean literacy in adults and children. Window on Washington Waters displays the outer coast marine environments and sea life of the Olympic Coast National Marine Sanctuary.
Video has become a key tool for scientific communication because it increases the outreach and impact of projects, furthers scientific research within and across fields of study, and offers an accessible medium to engage the public in the understanding of science. This project supports the expansion of an interactive, online STEM Videohall where hundreds of NSF-funded researchers share their work through brief video narratives and interactive discussion. While the Videohall is accessible year-round, periodic annual Showcase events are used to drive visitors to the site where they can engage with one another, the project investigators and trained facilitators. The Videohall is a multiplier of NSF's investments in individual projects because it allows STEM education researchers to become aware of, and learn from, related work that is funded across NSF programs and directorates, and other federal agencies. In 3-minute video narratives, investigators share ideas, resources, data, evidence of impact, strategies and challenges. The Videohall platform supports open access and is designed to foster communication in ways that scale beyond traditional formats such as academic conferences. Moreover, because the online STEM Videohall is open to the public, it allows STEM investigators to share their work with multiple stakeholder communities including K-12 educators and school leaders, informal educators and community organizations, the STEM industry, education policy makers and families. Finally, because each video narrative is accompanied by a facilitated online discussion thread, investigators have a unique and valuable mechanism for receiving feedback from these various stakeholder communities. The STEM Videohall project is funded by the Discovery Research K-12 program (DRK-12), which seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.
This project brings together seven NSF-funded resource centers who work with their respective communities to encourage Principal Investigators to share video narratives of their work through annual NSF STEM Showcase events. Each annual Showcase event presents videos created by 150-230 projects; many of the projects are affiliated with one of the seven resource centers, but others are from projects across NSF directorates and beyond. During a one-week period, Principal Investigators, their project staff, as well as the public at large, are invited to engage in interactive discourse, providing queries, comments, and feedback. Participants also vote for favorite presentations through "Presenters' Choice," "Public Choice," and "Facilitators' Choice," processes. This participant voting system serves to increase engagement and enhances outreach of the event through social media. After the one-week Showcase event concludes, all of the videos along with the related discourse remain available to the public online, who continue to access the Showcase throughout the year. Based on prior pilot work, it is estimated that over the course of a year, over 30,000 visitors, from over 150 countries, will engage with each annual Showcase. Videos from annual showcase events will be shared, reused, and repurposed to create new products with new constituencies. The project includes technical development efforts to iteratively improve its interactive platform, outreach efforts before each annual Showcase event, facilitation of the week-long event, and intensive dissemination efforts. A research component examines the extent of participation on various constituencies, the benefit of participation to projects, and the success of the events in terms of dissemination nationally and internationally.
To promote ocean and climate change literacy that addresses the needs of the Gulf of Mexico region, six aquariums based on the Gulf are partnering to educate the communities on both risks and ways to contribute locally to sustainability. The aquariums are combining community outreach programs, stewardship promotion, social networking and workshops for non-formal educators to reach broad audiences throughout the Gulf in Florida, Mississippi, Alabama, Louisiana, Texas and Mexico. Each partner is developing content that emphasizes local ecosystems and consequences of climate change, tailoring stewardship activities to address them. Over five years, the aquariums will reach out to diverse audiences in their communities, which have populations that total over five million people. With evaluated programs, exhibits, and websites in place, the partners will be able to continue building stewardship and educating their communities after the award ends.