The following summative report describes overarching evaluation findings from the evaluation of Leap into Science, including future considerations for the Leap into Science program team (program team). The report aims to summarize takeaways from 2018-2023 and report on overall insights pertaining to the core evaluation questions of interest.
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
Sciencenter will use a co-development process to strengthen rural engagement with hands-on and inquiry-based STEM for families and children. The museum will initiate the project in partnership with Moravia Central School District and Groton Public Library. The project team will also collaborate with advisors who have expertise in reaching rural audiences. Project activities will increase the museum's understanding of rural communities' needs around STEM. The museum will work with additional rural partners to develop activities and programs that meet identified needs. An external evaluator will track project progress and measure results. This evaluator will also train the museum's staff on data collection and recording methods. Data analysis throughout the project will support changes in programs and activities as needed.
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
Despite the rich scientific evidence of adaptations and their evolutionary basis, there are concerning public misconceptions about evolution, processes of natural selection, and adaptations in the biodiverse world. Such misconceptions begin early. Younger elementary school children are often resistant to the idea that one “kind” of animal could descend from a completely different kind of animal, and they see features as having always existed. Other misconceptions lead to an inaccurate belief that changes in individual organisms acquired in a lifetime are passed directly on to offspring or that entire populations transform as a whole. These cognitive biases and "intuitive” misunderstandings can persist into adulthood. This Innovations in Development project will counter that narrative through an informal science project focusing on the blue whale one of nature’s most spectacular stories of adaptation. It is a species that lives life at extremes: a long-distance migrator, a deep diver, an extravagant eater, the largest animal to ever exist. With its awe-inspiring size and rich mosaic of anatomical, physiological, and behavioral specializations, it serves as a bridge to an enriched understanding of universal concepts in elementary biology and can begin to dispel the deeply rooted misconceptions. The project deliverables include a giant screen film documenting the field work of research scientists studying the blue whales in the Indian Ocean and Gulf of Mexico; multi-platform educational modules and programs that will build on the blue whale content from the film for use in science center programs and rural libraries; and professional development webinars that will offer content utilization and presentation skills for ISE facilitators. Project partners include California Science Center, STAR Library Education Network, HHMI Tangled Bank Studios and SK Films.
The external evaluation studies will gather data from 20 participating rural libraries and 6 science museums. A formative evaluation of the film will be conducted in a giant screen theater setting with 75 families. After viewing a fine-cut version of the film they will complete age-appropriate post-viewing surveys on the film’s engagement, storytelling, content appeal and clarity, and learning value in communicating key science concepts. An external summative evaluation will include three studies. Study 1 will assess the implementation of the project at the 26 organizations, addressing the question: To what extent is the project implemented as envisioned in the libraries and science center settings? Baseline information will be collected, and later partners will complete post-grant surveys to report on their actual implementation of the project elements. In addition, the study will examine outcomes relating to professional development. Study 2 will be an evaluation of the film as experienced by 400 youth and parents in science centers and examining the question: To what extent does experiencing the film engage youth and parents and affect their interest, curiosity, and knowledge of blue whales, adaptations, and the scientific process? Study 3 will examine: To what extent and how does experiencing an educational module (virtual field trips, hands on activities, augmented reality) affect youth and parents’ interest, curiosity, and knowledge of adaptations and scientific process?
The Space Science Institute’s (SSI) National Center for Interactive Learning (NCIL), in partnership with the American Society for Civil Engineers (ASCE) and the University of Virginia (UVA), was awarded a grant from the National Science Foundation (NSF) to develop and implement a 3-year program, Project BUILD (Building Using an Interactive Learning Design). Project BUILD aims to bring together public library staff from six libraries (three rural and three urban) and professional engineers from ASCE to engage youth in grades 2-5 and their families in age-appropriate, technology-rich
This project will focus on addressing the challenges faced by rural youth with a particular emphasis on those youth who are English Language Learners. The project will provide informal education via libraries and librarians which can provide unique opportunities for rural youth and communities. Building on several years of research and experimentation, this project will augment the formal education sector, as well. The settings for the project are 12 rural school districts in largely Latinx communities. The project partners are the Space Science Institute, the American Library Association (ALA), the Institute for Learning Innovation and the Twin Cities Public Television. Expertise from the Latinx community will play a significant part in the project. The project will engage learners from diverse backgrounds, ages, and interests in science through a coordinated and tested strategy incorporating three Learning Pathways (i.e., Science Learning Spaces, Programs, and Science Kits) in a public library environment. The results should yield a model for Nationwide application.
The main goals are: 1) to establish learning pathways to engage rural communities through exhibit host libraries and (2) to increase art-rich STEM learning opportunities for rural communities through libraries and their support systems. Building on an established training model, the project will introduce library staff to the STEAM content of the exhibits and guide them in developing their own STEAM Learning Pathways. SciGirls digital media, hands-on activities, family resources, and a training network will expand the depth and reach of the project. The project draws on existing professional infrastructure to increase library staff capacity through ALA and the Institute's established community of practice. The researchers will study the efficacy of each pathway, alone and in tandem, on participant's interest development and persistence. The research will use a mixed-methods design-based approach that involves questionnaires, interviews and case studies.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Libraries can provide unique opportunities for rural youth and communities. Phase III of the STAR Library Network will be a collaboration with 12 rural school districts in largely Latinx communities to address the challenges faced by rural youth, particularly English Language Learners. The project will use a coordinated and tested strategy to establish three learning pathways in public libraries: science learning spaces with exhibits, library programs, and science kits. These resources will provide learners with art-rich STEM learning opportunities.
Partners
Project partners include the Space Science Institute, the American Library Association (ALA), the Institute for Learning Innovation, and Twin Cities Public Television. The project will rely significantly on expertise from the Latinx community.
Project Plan
Building on an established librarian training model, the project will introduce library staff to the STEAM content and guide them in developing their own STEAM Learning Pathways. The project will draw on existing professional infrastructure from the ALA and the Institute for Learning Innovation’s established community of practice. SciGirls digital media, hands-on activities, family resources, and a training network will expand the depth and reach of the project.
The Research
The research team will study the efficacy of each pathway, alone and in tandem, on participant’s interest development and persistence. The research will use a mixed-methods design-based approach that involves questionnaires, interviews, and case studies. The results should yield a model for nationwide application and contribute insights for the formal education sector.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This project will advance efforts of the Innovative Technology Experiences for Students and Teachers (ITEST) program to better understand and promote practices that increase students' motivations and capacities to pursue careers in fields of science, technology, engineering, or mathematics (STEM) by bringing together youth (grades 2-5), their families, librarians, and professional engineers in an informal environment centered on engaging youth with age-appropriate, technology-rich STEM learning experiences fundamental to the engineering design process. The overarching aim is to better understand how youth's learning preferences or dispositions relate to their STEM learning experiences. It also seeks to build community members' capacity to inspire and educate youth about STEM careers. The project team includes the Space Science Institute's (SSI) National Center for Interactive Learning (NCIL), the University of Virginia (UVA) and the American Society of Civil Engineers (ASCE). This team builds on the scope and reach of a prior NSF-funded project called the STAR Library Education Network (STAR_Net). As an extension of this prior work, Project BUILD will collaborate with 6 public libraries (3 urban and 3 rural) and their local ASCE Branches. Two libraries have been selected to serve as pilots: High Plains Public Library in Colorado and the African-American Research Library and Cultural Center in Florida. All partner libraries will develop a plan for recruiting participants from groups currently underrepresented in STEM professions. Project BUILD's specific aims are to 1) Engage underserved audiences, 2) Build the capacity of participating librarians and ASCE volunteers, 3) Increase interest and engagement in STEM activities for youth in grades 2-5 and their families, and 4) Conduct a comprehensive education research project. Program components include the following: 1) Community Dialogue Events, 2) a Professional Development Program for partner librarians and ASCE volunteers, and 3) Development of a Technology-rich Programming Kit and Circulating STEM Kit program. Two research questions will be addressed: 1) What common factors might identify youth who engage in project activities and what factors might differentiate between youth who continue with program engagement and those who do not? and 2) What programmatic factors (i.e. design and composition of program activities, library recruitment, librarian engagement, professional engineer engagement, etc.) might influence youth's initial and continued engagement in project activities as well as youth's reported future career interests? An external evaluation will investigate the quality of the project's process as well as its impact and effectiveness. Benefits to the participating libraries' communities, library and engineering professionals, and the education community will be achieved through 1) Community Dialogue events; 2) Library and Librarian Outreach; 3) ASCE Outreach; and 4) Publication of Research and Evaluation results.
This project develops and examines place-based learning using mobile augmented reality experiences for rural families where museums and science centers are scarce yet where natural resources are rich with outdoor trails, parks, and forestlands. The collaborative research team, with members from rural libraries, outdoor learning centers, learning scientists at Penn State University, and rural communities in Pennsylvania, will develop augmented reality and mobile learning resources for families and children aged from 4 to 12. The goal is to help people see what is not visible in real-time in order to learn about life and earth sciences based on local watersheds, trees, and seasonal cycles that are familiar and relevant to rural communities. To accomplish this goal, the project team will create scientifically meaningful experiences for rural families and children in their out-of-school time through three iterations of research and design. Although there is evidence that augmented reality can support learning, little empirical research has been conducted to determine what makes one type of augmented learning experience more effective than others in outdoor learning spaces. This project will produce research findings on the utility of augmented reality for science learning with families and youths outdoors. This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants
Through a four-year design-based research study, researchers will investigate three research questions. (1) How can outdoor learning experiences be enhanced with augmented reality and digital resources in ways that make science more visible and interesting?; (2) How do different forms of augmentations on trails and in gardens support science learning? 3) What social roles do children and parents play in supporting each other's science learning and connections to rural communities? Data collection includes video-recordings of children and families in the outdoors, learning analytics of people's behavior, and interviews with rural families. The project's research design will allow for the development of theory, which supports rural families learning science within and about their communities. At the end of the project, the team will offer generalizable design principles for technologically-enhanced informal learning for outdoor displays, gardens, and trails.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. This project will develop a national infrastructure of state and regional partnerships to scale up The Franklin Institute's proven model of Leap into Science, an outreach program that builds the capacity of children (ages 3-10) and families from underserved communities to participate in science where they live. Leap into Science combines children's science-themed books with hands-on science activities to promote life-long interest and knowledge of science, and does so through partnerships with informal educators at libraries, museums, and other out-of-school time providers. Already field-tested and implemented in 12 cities, Leap into Science will be expanded to 90 new rural and urban communities in 15 states, and it is estimated that this expansion will reach more than 500,000 children and adults as well as 2,700 informal educators over four years. The inclusion of marginalized rural communities will provide new opportunities to evaluate and adapt the program to the unique assets and needs of rural families and communities.
The project will include evaluation and learning research activities. Evaluation will focus on: 1) the formative issues that may arise and modifications that may enhance implementation; and 2) the overall effectiveness and impact of the Leap into Science program as it is scaled across more sites and partners. Learning research will be used to investigate questions organized around how family science interest emerges and develops among 36 participating families across six sites (3 rural, 3 urban). Qualitative methods, including data synthesis and cross-case analysis using constant comparison, will be used to develop multiple case studies that provide insights into the processes and outcomes of interest development as families engage with Leap into Science and a conceptual framework that guides future research. This project involves a partnership between The Franklin Institute (Philadelphia, PA), the National Girls Collaborative Project (Seattle, WA), Education Development Center (Waltham, MA), and the Institute for Learning Innovation (Corvallis, OR).
This project had three objectives to build knowledge with respect to advancing Informal STEM Education:
Plan, prototype, fabricate, and document a game-linked design-and-play STEM exhibit for multi-generational adult-child interaction utilizing an iterative exhibit design approach based on research and best practices in the field;
Develop and disseminate resources and models for collaborative play-based exhibits to the informal STEM learning community of practice of small and mid-size museums including an interactive, tangible tabletop design-and-play game and a related tablet-based game app for skateboarding science and technology design practice;
Conduct research on linkages between adult-child interactions and game-connected play with models in informal STEM learning environments.
Linked to these objectives were three project goals:
Develop tools to enable children ages 5-8 to collaboratively refine and test their own theories about motion by exploring fundamental science concepts in linked game and physical-object design challenge which integrates science (Newton’s Laws of Motion) with engineering (iterative design and testing), technology (computational models), and mathematics (predictions and comparisons of speed, distance, and height). [Linked to Objectives 1 & 3]
Advance the informal STEM education field’s understanding of design frameworks that integrate game environments and physical exhibit elements using tangibles and playful computational modeling and build upon the “Dimensions of Success” established STEM evaluation models. [Linked to Objectives 1 & 2]
Examine methods to strengthen collaborative learning within diverse families through opportunities to engage in STEM problem-based inquiry and examine how advance training for parents influences the extent of STEM content in conversations and the quality of interactions between caregivers and children in the museum setting. [Linked to Objectives 1 & 3]
The exhibit designed and created as a result of this grant project integrates skateboarding and STEM in an engaging context for youth ages 5 to 8 to learn about Newton’s Laws of Motion and connect traditionally underserved youth from rural and minority areas through comprehensive outreach. The exhibit design process drew upon research in the learning sciences and game design, science inquiry and exhibit design, and child development scholarship on engagement and interaction in adult-child dyads.
Overall, the project "Understanding Physics through Collaborative Design and Play: Integrating Skateboarding with STEM in a Digital and Physical Game-Based Children’s Museum Exhibit" accomplished three primary goals. First, we planned, prototyped, fabricated, and evaluated a game-linked design-and-play STEM gallery presented as a skatepark with related exhibits for adult-child interaction in a Children's Museum.
Second, we engaged in a range of community outreach and engagement activities for children traditionally underserved in Museums. We developed and disseminated resources for children to learn about the physics of the skatepark exhibit without visiting the Museum physically. For example, balance board activities were made portable, the skatepark video game was produced in app and web access formats, and ramps were created from block sets brought to off-site locations.
Third, we conducted a range of research to better understand adult-child interactions in the skatepark exhibit in the Children's Museum and to explore learning of physics concepts during physical and digital play. Our research findings collectively provide a new model for Children's Museum exhibit developers and the informal STEM education community to intentionally design, evaluate, and revise exhibit set-up, materials, and outcomes using a tool called "Dimensions of Success (DOS) for Children's Museum Exhibits." Research also produced a tool for monitoring the movement of children and families in Museum exhibit space, including time on task with exhibits, group constellation, transition time, and time in gallery. Several studies about adult-child interactions during digital STEM and traditional pretend play in the Museum produced findings about social positioning, interaction style, role, and affect during play.
DATE:
-
TEAM MEMBERS:
Deb DunkhaseKristen MissallBenjamin DeVane
The National Science Outreach Network will provide school children, teachers, and the general public with highly accessible interactive exhibits dealing with popular topics in science and technology. The network, initiated as a partnership between regional science centers and public libraries, will be modeled after the highly successful statewide Oregon Library Exhibits Network established in 1987. Through this smaller network, the Oregon Museum of Science and Industry, a nationally recognized pioneer in science exhibitry and outreach programming, circulates small hands-on exhibits to rural population centers through installations in public libraries, where school groups and families have free and convenient access. This national dissemination project will be initiated in five regional sites across the country (Colorado, Minnesota, New York, Tennessee, and Oregon) to further establish the model in rural, inner-urban, economically disadvantaged, and culturally diverse regions. With support from both the NSF and the regional networks, The National Science Outreach Network will design and duplicate six exhibits for circulation to dozens of local communities in each designated region. Over the next seven years, over six million individuals, many of whom do not currently frequent a local science center, will be introduced to popular science in a non-threatening, resource- rich setting. This will encourage further exploration and possible future visits to an accessible science center, and ultimately establish an ever-expanding network of museum and non-museum partners providing science and technology learning opportunities to millions of individuals each year.