The following summative report describes overarching evaluation findings from the evaluation of Leap into Science, including future considerations for the Leap into Science program team (program team). The report aims to summarize takeaways from 2018-2023 and report on overall insights pertaining to the core evaluation questions of interest.
In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
The Discovery Center at Murfree Spring, in partnership with six science centers and museums, will promote and invest in science education in rural communities with limited museum access. This coalition will work with two cohorts of rural school communities (12 total) and focus on engaging, learning from, and supporting rural school districts, teachers, families, and communities through relationship building, asset mapping, and the collaborative integration and implementation of museum resources. Additional activities include the production of publications, virtual presentations, and a virtual tool kit. The project will illustrate the ways in which museums can collaborate to support STEM and literacy at the K-2 level, enhance teacher self-efficacy, attitudes and beliefs, and engage family and community, strengthening services for Americans who live in the most rural areas.
ECHO, Leahy Center for Lake Champlain will increase its capacity to serve rural schools through programming opportunities under its STEM in Motion 2.0 program. In partnership with rural schools, they will conduct two year-long teacher institutes blending in-person and virtual professional development. They plan to develop a total of 270 in-person and virtual classroom STEM programs and produce 18 classroom curriculum kits and standard-activity aligned guides. As a result of STEM in Motion 2.0’s activities, the museum anticipate that 54 teachers will have additional capacity to deliver high-quality STEM learning experiences to K–5th grade students in underserved, rural communities.
Sciencenter will use a co-development process to strengthen rural engagement with hands-on and inquiry-based STEM for families and children. The museum will initiate the project in partnership with Moravia Central School District and Groton Public Library. The project team will also collaborate with advisors who have expertise in reaching rural audiences. Project activities will increase the museum's understanding of rural communities' needs around STEM. The museum will work with additional rural partners to develop activities and programs that meet identified needs. An external evaluator will track project progress and measure results. This evaluator will also train the museum's staff on data collection and recording methods. Data analysis throughout the project will support changes in programs and activities as needed.
The L.C. Bates Museum will deliver STEAM programming to neighboring rural, mostly low-income second grade children and their families through the Observing Ornithology science project. Over a two-year period, the museum will work with 40 teachers in 12 schools to support student learning tied to Next Generation core science performance measures. The project activities will use museum collections and other resources to present a series of three ornithology programs designed to motivate curiosity and engage children in observation activities that support a new approach to thinking, analyzing and solving. The museum will loan a new pop-up display to each of the 12 school libraries and will present four family and four children's museum bird days at the museum for participating students and their families during each year of the project. An external evaluator will measure the project's success in achieving defined performance measures that include strengthening the children's knowledge, understanding, and attitude toward the regional environment.
Hanohano o Oahu: The Geology and Moolelo of Kona to Ewa project will provide learning opportunities for 500 fourth grade students and their teachers from ten public schools located in central and leeward Oahu, Hawaii. A geology unit will be developed that includes a 90-minute class presentation, hands-on classroom activities, a Discovery Box to extend learning opportunities, and a full-day (5-hour) field trip experience. The multi-stop bus tour will be centered on the moku (district) of Kona and Ewa and highlight significant Oahu cultural sites, their moolelo (stories, history) and geology. A culture-based student activity booklet, hands-on activities, and other education materials will also be developed for the unit. The project will target rural communities with underserved families, large Hawaiian homestead neighborhoods, and little access to museum services. Participation in the programming will provide students and teachers with a better understanding of the connection between scientific information and Hawaiian knowledge.
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
Despite the rich scientific evidence of adaptations and their evolutionary basis, there are concerning public misconceptions about evolution, processes of natural selection, and adaptations in the biodiverse world. Such misconceptions begin early. Younger elementary school children are often resistant to the idea that one “kind” of animal could descend from a completely different kind of animal, and they see features as having always existed. Other misconceptions lead to an inaccurate belief that changes in individual organisms acquired in a lifetime are passed directly on to offspring or that entire populations transform as a whole. These cognitive biases and "intuitive” misunderstandings can persist into adulthood. This Innovations in Development project will counter that narrative through an informal science project focusing on the blue whale one of nature’s most spectacular stories of adaptation. It is a species that lives life at extremes: a long-distance migrator, a deep diver, an extravagant eater, the largest animal to ever exist. With its awe-inspiring size and rich mosaic of anatomical, physiological, and behavioral specializations, it serves as a bridge to an enriched understanding of universal concepts in elementary biology and can begin to dispel the deeply rooted misconceptions. The project deliverables include a giant screen film documenting the field work of research scientists studying the blue whales in the Indian Ocean and Gulf of Mexico; multi-platform educational modules and programs that will build on the blue whale content from the film for use in science center programs and rural libraries; and professional development webinars that will offer content utilization and presentation skills for ISE facilitators. Project partners include California Science Center, STAR Library Education Network, HHMI Tangled Bank Studios and SK Films.
The external evaluation studies will gather data from 20 participating rural libraries and 6 science museums. A formative evaluation of the film will be conducted in a giant screen theater setting with 75 families. After viewing a fine-cut version of the film they will complete age-appropriate post-viewing surveys on the film’s engagement, storytelling, content appeal and clarity, and learning value in communicating key science concepts. An external summative evaluation will include three studies. Study 1 will assess the implementation of the project at the 26 organizations, addressing the question: To what extent is the project implemented as envisioned in the libraries and science center settings? Baseline information will be collected, and later partners will complete post-grant surveys to report on their actual implementation of the project elements. In addition, the study will examine outcomes relating to professional development. Study 2 will be an evaluation of the film as experienced by 400 youth and parents in science centers and examining the question: To what extent does experiencing the film engage youth and parents and affect their interest, curiosity, and knowledge of blue whales, adaptations, and the scientific process? Study 3 will examine: To what extent and how does experiencing an educational module (virtual field trips, hands on activities, augmented reality) affect youth and parents’ interest, curiosity, and knowledge of adaptations and scientific process?
Although approximately one-quarter of U.S. students reside in rural communities, rural youth are fifty percent less likely to receive and engage in out-of-school STEM experiences than their urban counterparts. In addition, there has been significantly more investment in understanding and improving informal experiences in urban settings than in rural settings. As a result, there is less known about the characteristics of learning ecosystems and programs that support STEM learning for youth in informal contexts within rural communities. This Research in Service to Practice project aims to address this challenge by exploring the feasibility of a culturally relevant and sustaining STEM program designed specifically for rural youth and their families. Parents and caregivers play a critical role in fostering youths’ interests and persistence in STEM through their own engagement and by connecting them to STEM opportunities and STEM-related fields and career pathways. Through a partnership between the High Desert Museum in Oregon, the Institute for Learning Innovation, Maine Mathematics and Science Alliance, JKS Consulting, and three informal science education institutions, a year-long series of STEM-based workshops and experiences for youth and their families will be co-designed by members of the rural community, informal STEM educators, and STEM professionals and implemented within the rural communities of the participating informal science education institutions—Caddo Mounds State Historic Site Weeping Mary (TX), High Desert Museum (OR), Oregon Coast Aquarium, and The Wild Center (NY). Each series will reflect the cultural knowledge, connections, and resources specific to each rural community. In addition, the informal STEM educators and STEM professionals will receive training on facilitating the culturally sustaining workshops and experiences. Researchers at the Institution for Learning Innovation and the Maine Mathematics and Science Alliance together with the evaluator at JKS Consulting will employ a collaborative design-based research approach to identify and study the STEM learning practices and supports that occur within the program to promote youths’ interests and persistence in STEM. The findings will offer evidence-based insights to the field on how to better engage, reflect, and provide opportunities for diverse rural communities. Ultimately, this research has the potential to advance the current understanding thereby, strengthening rural STEM learning ecosystems and broadening STEM participation among youth in rural communities.
Over a four-year project duration, a collaborative design-based research approach will be employed to address the following research questions: (1) How does culturally sustaining informal STEM programming for families in rural communities contribute to increases in youth STEM persistence? (1a) How might this vary in relation to family and community factors? (2) How does culturally sustaining informal family STEM programming increase community connectivity between STEM-related resources and institutions across informal and formal learning contexts in rural communities leading to a more robust and inclusive STEM learning ecosystem? (2a) To what extent do participating families, informal STEM educators, STEM professionals, and community partners each play a role in increasing this connectivity? The research sample will include 300 families with youth ages 8-11, informal science educators, and STEM professionals across all four sites. Surveys, interviews and observations will be the primary data sources. Analysis of Variance and simple descriptive statistical analysis will be used to analyze the quantitative data. The qualitative data will be analyzed using thematic coding through NVivo. In addition, to complement the research data, JKS Consulting will conduct the formative and summative evaluations of the project to hone effective practices for training informal science learning practitioners in developing and implementing place-based, inquiry-based family learning in rural communities and effective and sustainable practices for engaging rural families in place-based STEM. Findings from the research will be made available and widely distributed in publications, conference presentations, and a multi-part Research to Practice Toolkit designed for parents and caregivers, informal science educators, STEM professionals, and the informal education field at large.
This Research in Service to Practice project is funded by the Advancing Informal STEM Learning (AISL) program.
Museums in the US receive approximately 55 million visits each year from students in school groups. Field trip visits to an art museum have been found to positively impact critical thinking skills, empathy and tolerance - an increase that can be even more significant for youth from rural or high-poverty regions. While field trips are popular, especially at science museums, there have been no experimental studies about their impact on STEM career choices and interests, much less any which used a culturally sensitive and responsive approach. Given the resources put into field trips, this study investigates if causal links can be drawn between museum experiences and impact on youth. The Museum of Science & Industry uses a Learning Labs approach for engaging its visitors. These "Learning Labs" are facilitated experiences that run roughly an hour. Currently there are 12 lab topics. This study focuses on MedLab--one of the learning labs--as the setting for the research. MedLab is designed for on-site and online experience using ultra-sophisticated and highly versatile technology in challenges taken from research on the top healthcare issues that face adolescents in their communities.
This study is informed by research and theory on Social Cognitive Career Theory (SCCT) and Racial and Ethnic Identity. The former describes a process many follow when thinking about career options, broadly. The latter describes how people see themselves in the world through their membership with a racial and/or ethnic group. Both processes can collectively influence STEM career choices. This study follows an embedded mixed-method design. The quantitative portion includes an experimental, pre/post/delayed post-test design of both educators and their students using multiple measures taken mostly from previously published instruments. The qualitative portion includes observation rubrics of MedLab sessions along with interviews and focus groups with staff, educators, students and families that take place both within and outside of the museum. This is an experimental study of moderate size of both heterogeneous teacher and student populations in real world settings. It involves comparing youth and educators that participate in MedLab with those who do not. By conducting research that looks at each community through the lens of their unique experiences, the research will measure their impact more sensitively and authentically, addressing a gap in current literature on informal science, technology, engineering, or mathematics (STEM) career education with diverse students.
This study is funded by the Advancing Informal STEM Learning (AISL) program and the Innovative Technology Experiences for Students and Teachers (ITEST) program.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. This project will develop a national infrastructure of state and regional partnerships to scale up The Franklin Institute's proven model of Leap into Science, an outreach program that builds the capacity of children (ages 3-10) and families from underserved communities to participate in science where they live. Leap into Science combines children's science-themed books with hands-on science activities to promote life-long interest and knowledge of science, and does so through partnerships with informal educators at libraries, museums, and other out-of-school time providers. Already field-tested and implemented in 12 cities, Leap into Science will be expanded to 90 new rural and urban communities in 15 states, and it is estimated that this expansion will reach more than 500,000 children and adults as well as 2,700 informal educators over four years. The inclusion of marginalized rural communities will provide new opportunities to evaluate and adapt the program to the unique assets and needs of rural families and communities.
The project will include evaluation and learning research activities. Evaluation will focus on: 1) the formative issues that may arise and modifications that may enhance implementation; and 2) the overall effectiveness and impact of the Leap into Science program as it is scaled across more sites and partners. Learning research will be used to investigate questions organized around how family science interest emerges and develops among 36 participating families across six sites (3 rural, 3 urban). Qualitative methods, including data synthesis and cross-case analysis using constant comparison, will be used to develop multiple case studies that provide insights into the processes and outcomes of interest development as families engage with Leap into Science and a conceptual framework that guides future research. This project involves a partnership between The Franklin Institute (Philadelphia, PA), the National Girls Collaborative Project (Seattle, WA), Education Development Center (Waltham, MA), and the Institute for Learning Innovation (Corvallis, OR).
Chemistry is an important and widely relevant field of science. However, when compared with other STEM content areas, chemistry is under-represented in U.S. science museums and other informal educational environments. This project will build, and build knowledge about, innovative approaches to delivering informal science learning activities in chemistry. The project will not only increase public interest and understanding of chemistry but also increase public perception of chemistry's relevance and increase the public's self-efficacy with respect to chemistry. This project outcomes will include a guide for practitioners along with activity materials that will be packaged into a kit, distributed, and replicated for use by informal science educators, chemists, and chemistry students at 250 sites across the U.S. The project team will reach out to organizations that serve diverse audiences and diverse geographic locations, including organizations in rural and inner-city areas. The kits will provide guidance on engaging girls, people with various abilities, Spanish speakers, and other diverse audiences, and include materials in Spanish. Written guides, training videos, and training slides will be included to support training in science communication in general, as well as chemistry in particular. This project is supported by the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings, as a part of its overall strategy to enhance learning in informal environments.
This project will take an innovative approach to develop informal educational activities and materials about chemistry. Rather than starting with content goals, the project will start with a theoretical framework drawn from research about affecting attitudes about science related to interest, relevance, and self-efficacy. A design-based research approach (DBR) will be used to apply that framework to the development of hands-on educational activities about chemistry, while also testing and modifying the framework itself. (DBR blends empirical educational research with the theory-driven design of learning environments.) Existing or new educational activities that appear to embody key characteristics defined in the framework will be tested with public audiences for their impact on visitors. Researchers and educators will determine how different characteristics of the educational activities defined in the framework affect the outcomes. The activities will be modified and tested iteratively until the investigators achieve close alignment between framework and impacts.. The project team will continue the design-based research approach both to examine groups of activities in which synergies can have impacts beyond single interactions as well as to examine varied ways of training facilitators who can also significantly affect outcomes. In this way, the project will generate knowledge about how kits of hands-on informal learning activities can stimulate attitudes of interest, relevance, and self-efficacy with respect to the neglected field of chemistry. The project teams will broadly disseminate project outcomes within the educational research, science and informal Science, Technology, Engineering and Mathematics (STEM) education communities. While this project will focus on chemistry, the strategies it will develop and test through a design-based research process will provide valuable insight into effective approaches for informal STEM education more broadly.