The broad purpose of this project is to contribute to capacity-building efforts—to strengthen the evaluation of programs and learning experiences in informal science education (ISE). We focus on improving the quality of summative evaluations, which have been called upon to inform decision-making and practice, contribute knowledge to the field, and help make the case for the value of informal learning. We previously developed a framework that synthesizes key elements of a high-quality summative evaluation. The framework has three dimensions: (a) examine the underlying rationale of the program, exhibition, experience, or intervention being evaluated; (b) balance methodological rigor with sensitivity to the informal context; and (c) prioritize uses of the evaluation by addressing stakeholders’ needs. Evaluators may draw on all three dimensions to provide summary judgments on the value of what was evaluated. We extend our work in two ways: (1) Examine how the framework could be used as a guiding lens for planning or conducting future or current summative evaluations. We will work with partners in informal science institutions to support and document their evaluation activities along the dimensions of evaluation quality, with the intent of improving the framework and creating concrete exemplars of its application in practice. (2) Conceptualize alternative models for enhancing professional training in ISE evaluation. We will research current programs and practices for building technical evaluation capacity, looking for promising and innovative approaches that include apprenticeship or “hands-on” experiences. This 18-month project (January 2015 through June 2016) is funded by the Gordon and Betty Moore Foundation.
This monograph has been created by the TEAMS (Traveling Exhibits At Museums of Science) Collaborative, a group of seven small U.S. science centers, to share experiences, observations, and lessons learned with the broader science museum field. Our intention is to help others who might be interested in forming a collaborative to work though some key issues, most of which have to do with the relationships between collaborative members, rather than more technical matters. We hope that this will provide a vision for others as to how collaboration among science centers can work. The monograph looks
DATE:
TEAM MEMBERS:
Charles TrautmannMark St. JohnDavid Goudy
Goodman Research Group, Inc., (GRG), Cambridge, MA, conducted the formative evaluation of The Music Instinct project. The NSF-funded project aims to bring to PBS viewers the strong evidence of the connections between music and science, as well as to facilitate a deeper understanding of both fields. The Music Instinct project, presented by WNET/Thirteen, in collaboration with Mannes Productions, includes a two-hour television program, a website, and ancillary educational materials. The purpose of the formative evaluation is to obtain timely information to support and guide the producers as they
DATE:
TEAM MEMBERS:
Rucha LondheMiriam KochmanNivedita RanadeIrene F GoodmanWNET/ThirteenMannes Productions Inc.
With the Role of Media in Supporting Informal Science Learning project, the Institute for Learning Innovation (ILI) and Grunwald Associates sought to establish a “national learning initiative” to explore the intersection of media and informal science learning. To do so, ILI proposed an initial conference followed by the development of a website and online community. The National Science Foundation funded this project, with additional funding provided by the National Parks Service. Held in March 2009, the 1.5 day conference was designed to be a “first step” in the development of a conversation
DATE:
TEAM MEMBERS:
Elizabeth Bandy, Ph.D.Institute for Learning Innovation
The National Partnerships for Afterschool Science (NPASS) Project was led by The Center for Science Education (CSE) at Education Development Center, Inc. (EDC) in Newton, MA in partnership with the Lawrence Hall of Science (LHS) in Berkeley, CA, with funding from the National Science Foundation (NSF). The NPASS project approach relied on a professional development training and mentorship model to build the capacity of community-based organizations (CBOs) to lead high-quality, hands-on science and engineering activities in their afterschool programming. An overarching project research question
The TEAMS Collaborative received three separate rounds of funding from the National Science Foundation; Inverness Research served as the external evaluator to the project for all three rounds of funding. This summative report details the return on NSF's long-term investment in the TEAMS Collaborative. It outlines the overall contributions of the project to participating museums, individual staff at those museums, and to the larger field of informal science education institutions. It also summarizes the ways in which the TEAMS Collaborative was able to effectively capitalize on NSF's investment
This model science teacher retention and mentoring project will involve more than 300 elementary teachers in "Lesson Study" of inquiry science around school gardens. Drawing on the rich resources of the University of California Botanical Garden and the science educators at the Lawrence Hall of Science this project will develop Teacher Leaders and provide science content professional development to colleagues in four urban school districts. Using the rich and authentic contexts of gardens to engage students and teachers in scientific inquiry opens the opportunity to invite parents to become actively involved with their children in the learning process. As teachers improve their classroom practices of teaching science through inquiry with the help of school-based mentoring they are able to connect the teaching of science to mathematics and literacy and will be able to apply the lesson study approach in their teaching of other innovative projects. Teacher leaders and mentors will have on-going learning opportunities as well as engage participating teachers in lesson study and reflection aimed toward improving science content understanding and the quality of science learning in summer garden learning experiences and having context rich science inquiry experiences throughout the school year.
Following a 2011 report by the National Research Council (NRC) on successful K-12 education in science, technology, engineering, and mathematics (STEM), Congress asked the National Science Foundation to identify methods for tracking progress toward the report's recommendations. In response, the NRC convened the Committee on an Evaluation Framework for Successful K-12 STEM Education to take on this assignment. The committee developed 14 indicators linked to the 2011 report's recommendations. By providing a focused set of key indicators related to students' access to quality learning, educator's
DATE:
TEAM MEMBERS:
National Research Council
resourceresearchProfessional Development, Conferences, and Networks
This guide provides effective practices for anyone — university faculty member, K–12 teacher, or administrator — who wants to create a project that partners science, technology, engineering, and mathematics (STEM) graduate students (Fellows) with K–12 teachers on a sustained basis. These recommendations come from the community of faculty members, graduate students, K–12 teachers, program managers, and evaluators who participated in the U.S. National Science Foundation (NSF) Graduate STEM Fellows in K–12 Education (GK–12) Program from its start in 1999 through 2012. The guide was written to
The intent of this project is to use social network methods to study networks of afterschool and informal science stakeholders. It would attempt to create knowledge that improves afterschool programs access to informal science learning materials. This is an applied research study that applies research methods to improving access to and enactment of informal science education programs across a range of settings. The investigators plan to collect data from 600 community- and afterschool programs in California, conduct case studies of 10 of these programs, and conduct surveys of supporting intermediary organizations. The analysis of the data will provide descriptions of the duration, intensity, and nature of the networks among afterschool programs and intermediary agencies, and the diffusion patterns of science learning materials in afterschool programs. The project will yield actionable knowledge that will be disseminated among afterschool programs, intermediary organizations, funding agencies, and policymakers to improve the dissemination and support of afterschool science learning opportunities. The project is focused on free-choice settings where every day the largest numbers of children attend afterschool programs at schools and in other community settings. It seeks information about what conditions are necessary for informal science programs to significantly impact the largest possible number of children in these settings.
DATE:
-
TEAM MEMBERS:
Barbara MeansAnn HouseCarlin LlorenteRaymond McGhee
The University of Massachusetts Lowell conducted 1.5-day conference in the fall of 2011, titled "Learning on the Go: Using Out-of-Home Media to Communicate Climate Science." The conference, held at the Lowell Inn and Conference Center, brought together approximately 125 professionals and students in climate science, communications, out-of-home media, social science, informal and formal science education, and educational psychology with the goal of exploring opportunities for applying out-of-home media to communicating science to the public, with a particular emphasis on climate change science. "Out-of-home media" is defined as any type of communication that reaches individuals while they are out of the home, including mobile media, billboards, mass transit placards, posters, etc. The intent was to consider how informal science education and its impacts on learning can be expanded via the adaptation of such media to the goals of ISE. Conference proceedings and podcasts of keynote sessions will be made available on a conference Web site. Conference evaluation will be conducted by Arbor Consulting Partners.
Living Laboratory® (developed at the Museum of Science, Boston in 2005) is a new model for partnerships between museums and cognitive scientists, bringing cognitive scientists to museums, where they conduct active research studies with museum visitors as their subjects. In 2011, the Museum of Science began scaling up Living Laboratory to create a National Living Lab network. In Year 1, the program expanded to three new Hub sites: Madison Children’s Museum, Maryland Science Center, and Oregon Museum of Science and Industry. This report summarizes all formative evaluation from Year 1 of the