The National Building Museum (NBM) contracted RK&A, Inc. to conduct a summative evaluation of the Why Engineering? distance learning program. The goal of the evaluation was to assess program operations and explore the extent to which the program achieved its intended outcomes for students and teachers.
How did we approach this study?
RK&A used three methodologies for the study: online program observations; student assessments administered immediately after the program; and telephone interviews with teachers. Observations were primarily used to gain a holistic understanding of how the
RK&A conducted a summative evaluation of five climate change and resiliency programs at the Science Museum of Virginia (SMV). The summative evaluation was the culmination of a three-year grant from the National Oceanic and Atmospheric Administration (NOAA)—front-end and formative evaluations were completed in the two years prior. The goal of this evaluation was to test the extent to which the programs achieved the intended visitor outcomes defined in SMV’s Impact Framework.
How did we approach this study?
The summative evaluation focused on five programs—the Digital Dome theater’s
Informal STEM education (ISE) organizations, especially museums, have used evaluation productively but unevenly. We argue that advancing evaluation in ISE requires that evaluation capacity building (ECB) broadens to include not only professional evaluators but also other professionals such as educators, exhibit developers, activity facilitators, and institutional leaders. We identify four categories of evaluation capacity: evaluation skill and knowledge, use of evaluation, organizational systems related to conducting or integrating evaluation, and values related to evaluation. We studied a
Informal STEM education institutions seek to engage broader cross sections of their communities to address inequities in STEM participation and remain relevant in a multicultural society. In this chapter, we advance the role that evaluation can play in helping the field adopt more inclusive practices and achieve greater equity than at present through evaluation that addresses sociopolitical contexts and reflects the perspectives and values of non-dominant communities. To do this for specific projects, we argue that evaluation should privilege the voices and lived experiences of non-dominant
The free‐choice nature of informal STEM education (ISE) makes rigorous and contextually appropriate evaluation of outcomes challenging. Traditional measures such as surveys and interviews have been widely used in ISE evaluations, but they have limitations: They are typically self‐reports that are susceptible to the reactive effects of measurement, and they tend to intrude upon the participant's learning experience. The ISE field needs measures that capture outcomes in more direct and less obtrusive ways, permitting triangulation with multiple measures on outcomes. In this chapter, we define
We define "informal STEM education" and explain some of the reasons its outcomes are so inherently challenging to evaluate, including the critical need for ecological validity and the fact that many informal learning experiences are low-visibility and opportunistic. We go on to highlight significant advances in the field, starting with the fundamental embracing of learning outcomes that go well beyond narrow measures of knowledge and skills, to include interest, engagement, and identity-building. Within that framework, we note the development of shared constructs and shared instruments
The STEM Interest and Engagement (STEM IE) Study was a four-year project funded by the National Science Foundation under the auspices of its Advancing Informal STEM Learning (AISL) program that was designed to better understand what types of practices, supports, and opportunities afforded to early adolescent youth: (1) Are especially effective in helping youth experience in-the-moment engagement while participating in ISL activities, and (2) Serve to support growth in STEM interest and aspirations. The study was conducted in a total of nine, STEM-oriented, summer learning programs serving
DATE:
TEAM MEMBERS:
Neil NaftzgerJennifer SchmidtLee ShumowPatrick BeymerJoshua Rosenberg
Parents are vital players in raising youth’s awareness of the value of STEM and in brokering their participation in activities that build STEM competencies.
STEM Next Opportunity Fund is committed to ensuring that every child – especially girls, youth of color, kids in low-income communities, and youth with disabilities – has access to STEM experiences and the social capital that lead to greater opportunities in academics and careers. We believe family engagement is a game changer and offer this white paper to raise awareness of its importance and amplify promising practices.
‘6 Degrees of Connection’ is an informative and memorable program that combines a one-hour Science On a Sphere® interactive presentation with a follow-up creative art activity – each aspect of which encourages middle school students to think of interconnections among phenomena on the planet, including the natural world and human activity.
The ‘6 Degrees of Connection’ program was developed by the Nurture Nature Center (Easton PA) in collaboration with the Maryland Science Center (Baltimore MD) through an extensive process of developing the concept, prototypes, and final program. Lehigh
The HOWL science team set out to analyze the quality of the Hofmann Forest’s watershed in order to understand the Hofmann’s place in the coastal ecosystem as a whole and how its quality would affect the surrounding human and ecological community. However, we realized quickly that we would need help collecting hundreds of stream water samples and observations, and so we partnered with the Izaak Walton League to help us contact and get local community members involved in such an overwhelming task. Local volunteers will not only help collect samples for this large-scale citizen science project, but will also help us try out some new equipment for water sampling and testing. And so, along with your help, the HOWL Project will:
Gain knowledge of water quality, and quantity, of the White Oak, New, and Trent Rivers in the Hofmann Forest in Eastern NC.
Demonstrate the feasibility of using low-cost analytical equipment for water quality testing.
Increase understanding of North Carolina’s rapidly changing coast due to the threats of sea-level rise, deforestation, agricultural expansion, and new substantial developments.
DATE:
-
TEAM MEMBERS:
Meredith HovisFrederick CubbageMadhusudan KattiKathleen McGinley
The Collaboration for Ongoing Visitor Experience Studies (COVES) aims to help science centers gather and share data to better understand visitors’ experiences. For the summative evaluation of COVES, the Museum of Science’s Research and Evaluation Department studied the program’s impacts on participating museums and museum professionals. Specifically, this evaluation was designed to:
Examine participants’ overall satisfaction with the collaboration and their likelihood to recommend it to others;
Understand the impact that COVES is having on Participating Institutions;
Assess whether
Citizen science, also known as participatory research, combines the efforts among professional researchers and community volunteers to collect data. We have established a collaborative project in eastern North Carolina, near the 79,000-acre Hofmann Forest, comprising of 55,000 acres of planted forests and 24,00 acres of deep pocosin natural forests. The White Oak River, New River, and Trent River all flow out of the Hofmann. The Hofmann acts plays a keystone ecological role as it acts as a natural filtration system for harmful runoff that occurs in the coastal plain of North Carolina.
The
DATE:
TEAM MEMBERS:
Meredith HovisFrederick CubbageMadhusudan KattiKathleen McGinley