The broad purpose of this project is to contribute to capacity-building efforts—to strengthen the evaluation of programs and learning experiences in informal science education (ISE). We focus on improving the quality of summative evaluations, which have been called upon to inform decision-making and practice, contribute knowledge to the field, and help make the case for the value of informal learning. We previously developed a framework that synthesizes key elements of a high-quality summative evaluation. The framework has three dimensions: (a) examine the underlying rationale of the program, exhibition, experience, or intervention being evaluated; (b) balance methodological rigor with sensitivity to the informal context; and (c) prioritize uses of the evaluation by addressing stakeholders’ needs. Evaluators may draw on all three dimensions to provide summary judgments on the value of what was evaluated. We extend our work in two ways: (1) Examine how the framework could be used as a guiding lens for planning or conducting future or current summative evaluations. We will work with partners in informal science institutions to support and document their evaluation activities along the dimensions of evaluation quality, with the intent of improving the framework and creating concrete exemplars of its application in practice. (2) Conceptualize alternative models for enhancing professional training in ISE evaluation. We will research current programs and practices for building technical evaluation capacity, looking for promising and innovative approaches that include apprenticeship or “hands-on” experiences. This 18-month project (January 2015 through June 2016) is funded by the Gordon and Betty Moore Foundation.
This monograph has been created by the TEAMS (Traveling Exhibits At Museums of Science) Collaborative, a group of seven small U.S. science centers, to share experiences, observations, and lessons learned with the broader science museum field. Our intention is to help others who might be interested in forming a collaborative to work though some key issues, most of which have to do with the relationships between collaborative members, rather than more technical matters. We hope that this will provide a vision for others as to how collaboration among science centers can work. The monograph looks
DATE:
TEAM MEMBERS:
Charles TrautmannMark St. JohnDavid Goudy
MarshAccess is an informal science education program based at the Meadowlands Environment Center (MEC) in Lyndhurst, NJ, and funded by the National Science Foundation. MarshAccess seeks to engage largely underserved populations of young and older adults with disabilities, as well as older adults with age-related limitations, in outdoor experiential STEM activities centered on the New Jersey Meadowlands marsh ecosystem. Program modules are designed to increase interest in science, increase scientific literacy, develop a sustained relationship between the MEC and the target audience and audience
DATE:
TEAM MEMBERS:
Hilarie B. DavisBradford T. DaveyRamapo College of New Jersey
This research was commissioned to explore visitors’ experiences in When the Dinosaurs Were Gone (later titled "When Crocodiles Ruled", a temporary exhibition at the Science Museum of Minnesota. As a summative evaluation, the research sought to address several issues and objectives: 1. overall impressions of the exhibition — visitors’ ratings of enjoyment, interest in the information, what they liked most, and what changes they would suggest (or not want) for a traveling version of this exhibit; 2. extent of use of the exhibition — amount of time spent in the exhibition and the extent to which
DATE:
TEAM MEMBERS:
People Places & Design ResearchScience Museum of MinnesotaJolene Hart
The goals of this Pathways Project will be to: 1) learn how to better recruit and sustain deep relationships with seniors and youth; 2)facilitate the roles that these two audiences can play as collaborators in field research and conservation science; 3) study ways that seniors and young people as well as scientists and lay people might interact most effectively while in training and in the field, and 4) study cognitive and affective impacts of such collaborations upon both citizens and professionals.
DATE:
TEAM MEMBERS:
PEER AssociatesNational Audubon Society
Youth Radio, an after-school media production program, consists of a main campus in Oakland, CA, and regional bureaus in Atlanta, GA, Washington, D.C., and Los Angeles, CA. Youth Radio’s model is to engage underserved young people in broadcast journalism, radio and web production, engineering, and media literacy through media projects that are relevant to the students’ lives and communities. In doing so, Youth Radio prepares young people for college programs and careers in media. The organization also supports young people in their transition from school to career through an externship program
Evaluators sought to determine impact on math attitude and content knowledge of students (3rd - 6th grade), and math literacy workers (high school and college students), based on pre-post testing and observation of YPP after school programs, in which college and high school students teach math games to elementary and middle school students in marginalized and vulnerable communities. The study focused on the Chicago YPP site, 1 of 7 in the YPP national network. Increases in math attitude scores were not statistically significant, however in some instances evaluators found significant increases
DATE:
TEAM MEMBERS:
Norman LedermanYoung People's Project, Inc.
Goodman Research Group, Inc., (GRG), Cambridge, MA, conducted the formative evaluation of The Music Instinct project. The NSF-funded project aims to bring to PBS viewers the strong evidence of the connections between music and science, as well as to facilitate a deeper understanding of both fields. The Music Instinct project, presented by WNET/Thirteen, in collaboration with Mannes Productions, includes a two-hour television program, a website, and ancillary educational materials. The purpose of the formative evaluation is to obtain timely information to support and guide the producers as they
DATE:
TEAM MEMBERS:
Rucha LondheMiriam KochmanNivedita RanadeIrene F GoodmanWNET/ThirteenMannes Productions Inc.
This summative evaluation of the exhibition Robots & Us was designed to investigate how visitor audiences used and experienced this exhibition in relation to the project’s objectives and challenges. Visitors’ expectations and perceptions in relation to the project’s content goals prompted the summative evaluation to focus on specific challenges including: attitudes and perceptions about technology, connections between robots and people, appeal to a broad audience, and reactions to specific exhibits.
DATE:
TEAM MEMBERS:
Jeff HaywardJolene HartScience Museum of Minnesota
With the Role of Media in Supporting Informal Science Learning project, the Institute for Learning Innovation (ILI) and Grunwald Associates sought to establish a “national learning initiative” to explore the intersection of media and informal science learning. To do so, ILI proposed an initial conference followed by the development of a website and online community. The National Science Foundation funded this project, with additional funding provided by the National Parks Service. Held in March 2009, the 1.5 day conference was designed to be a “first step” in the development of a conversation
DATE:
TEAM MEMBERS:
Elizabeth Bandy, Ph.D.Institute for Learning Innovation
The National Partnerships for Afterschool Science (NPASS) Project was led by The Center for Science Education (CSE) at Education Development Center, Inc. (EDC) in Newton, MA in partnership with the Lawrence Hall of Science (LHS) in Berkeley, CA, with funding from the National Science Foundation (NSF). The NPASS project approach relied on a professional development training and mentorship model to build the capacity of community-based organizations (CBOs) to lead high-quality, hands-on science and engineering activities in their afterschool programming. An overarching project research question
Formative evaluation of one of four pieces of the Marcellus Matters: EASE project. This study examined the effectiveness of an eight-week adult/community education program about topics related to natural gas development.