The following summative report describes overarching evaluation findings from the evaluation of Leap into Science, including future considerations for the Leap into Science program team (program team). The report aims to summarize takeaways from 2018-2023 and report on overall insights pertaining to the core evaluation questions of interest.
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
Despite the rich scientific evidence of adaptations and their evolutionary basis, there are concerning public misconceptions about evolution, processes of natural selection, and adaptations in the biodiverse world. Such misconceptions begin early. Younger elementary school children are often resistant to the idea that one “kind” of animal could descend from a completely different kind of animal, and they see features as having always existed. Other misconceptions lead to an inaccurate belief that changes in individual organisms acquired in a lifetime are passed directly on to offspring or that entire populations transform as a whole. These cognitive biases and "intuitive” misunderstandings can persist into adulthood. This Innovations in Development project will counter that narrative through an informal science project focusing on the blue whale one of nature’s most spectacular stories of adaptation. It is a species that lives life at extremes: a long-distance migrator, a deep diver, an extravagant eater, the largest animal to ever exist. With its awe-inspiring size and rich mosaic of anatomical, physiological, and behavioral specializations, it serves as a bridge to an enriched understanding of universal concepts in elementary biology and can begin to dispel the deeply rooted misconceptions. The project deliverables include a giant screen film documenting the field work of research scientists studying the blue whales in the Indian Ocean and Gulf of Mexico; multi-platform educational modules and programs that will build on the blue whale content from the film for use in science center programs and rural libraries; and professional development webinars that will offer content utilization and presentation skills for ISE facilitators. Project partners include California Science Center, STAR Library Education Network, HHMI Tangled Bank Studios and SK Films.
The external evaluation studies will gather data from 20 participating rural libraries and 6 science museums. A formative evaluation of the film will be conducted in a giant screen theater setting with 75 families. After viewing a fine-cut version of the film they will complete age-appropriate post-viewing surveys on the film’s engagement, storytelling, content appeal and clarity, and learning value in communicating key science concepts. An external summative evaluation will include three studies. Study 1 will assess the implementation of the project at the 26 organizations, addressing the question: To what extent is the project implemented as envisioned in the libraries and science center settings? Baseline information will be collected, and later partners will complete post-grant surveys to report on their actual implementation of the project elements. In addition, the study will examine outcomes relating to professional development. Study 2 will be an evaluation of the film as experienced by 400 youth and parents in science centers and examining the question: To what extent does experiencing the film engage youth and parents and affect their interest, curiosity, and knowledge of blue whales, adaptations, and the scientific process? Study 3 will examine: To what extent and how does experiencing an educational module (virtual field trips, hands on activities, augmented reality) affect youth and parents’ interest, curiosity, and knowledge of adaptations and scientific process?
Through Project BUILD, a STAR Library Network (STAR Net) program funded by the National Science Foundation, the American Society of Civil Engineers (ASCE) and the Space Science Institute’s National Center for Interactive Learning (NCIL) offered the virtual Dream, Build, Create program which consisted of (1) the award-winning documentary Dream Big: Engineering Our World and (2) five live-streamed panels of diverse engineers (Dream Teams) who shared their stories of what it means to be an engineer.
The external evaluation, conducted by Education Development Center (EDC), aimed to examine how
The Space Science Institute’s (SSI) National Center for Interactive Learning (NCIL), in partnership with the American Society for Civil Engineers (ASCE) and the University of Virginia (UVA), was awarded a grant from the National Science Foundation (NSF) to develop and implement a 3-year program, Project BUILD (Building Using an Interactive Learning Design). Project BUILD aims to bring together public library staff from six libraries (three rural and three urban) and professional engineers from ASCE to engage youth in grades 2-5 and their families in age-appropriate, technology-rich
This project will focus on addressing the challenges faced by rural youth with a particular emphasis on those youth who are English Language Learners. The project will provide informal education via libraries and librarians which can provide unique opportunities for rural youth and communities. Building on several years of research and experimentation, this project will augment the formal education sector, as well. The settings for the project are 12 rural school districts in largely Latinx communities. The project partners are the Space Science Institute, the American Library Association (ALA), the Institute for Learning Innovation and the Twin Cities Public Television. Expertise from the Latinx community will play a significant part in the project. The project will engage learners from diverse backgrounds, ages, and interests in science through a coordinated and tested strategy incorporating three Learning Pathways (i.e., Science Learning Spaces, Programs, and Science Kits) in a public library environment. The results should yield a model for Nationwide application.
The main goals are: 1) to establish learning pathways to engage rural communities through exhibit host libraries and (2) to increase art-rich STEM learning opportunities for rural communities through libraries and their support systems. Building on an established training model, the project will introduce library staff to the STEAM content of the exhibits and guide them in developing their own STEAM Learning Pathways. SciGirls digital media, hands-on activities, family resources, and a training network will expand the depth and reach of the project. The project draws on existing professional infrastructure to increase library staff capacity through ALA and the Institute's established community of practice. The researchers will study the efficacy of each pathway, alone and in tandem, on participant's interest development and persistence. The research will use a mixed-methods design-based approach that involves questionnaires, interviews and case studies.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Libraries can provide unique opportunities for rural youth and communities. Phase III of the STAR Library Network will be a collaboration with 12 rural school districts in largely Latinx communities to address the challenges faced by rural youth, particularly English Language Learners. The project will use a coordinated and tested strategy to establish three learning pathways in public libraries: science learning spaces with exhibits, library programs, and science kits. These resources will provide learners with art-rich STEM learning opportunities.
Partners
Project partners include the Space Science Institute, the American Library Association (ALA), the Institute for Learning Innovation, and Twin Cities Public Television. The project will rely significantly on expertise from the Latinx community.
Project Plan
Building on an established librarian training model, the project will introduce library staff to the STEAM content and guide them in developing their own STEAM Learning Pathways. The project will draw on existing professional infrastructure from the ALA and the Institute for Learning Innovation’s established community of practice. SciGirls digital media, hands-on activities, family resources, and a training network will expand the depth and reach of the project.
The Research
The research team will study the efficacy of each pathway, alone and in tandem, on participant’s interest development and persistence. The research will use a mixed-methods design-based approach that involves questionnaires, interviews, and case studies. The results should yield a model for nationwide application and contribute insights for the formal education sector.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This poster, which was presented in Alexandria, VA at the CAISE AISL PI meeting in February 2019, summarizes the Remake Making professional development program for makerspace facilitation.
This project will advance efforts of the Innovative Technology Experiences for Students and Teachers (ITEST) program to better understand and promote practices that increase students' motivations and capacities to pursue careers in fields of science, technology, engineering, or mathematics (STEM) by bringing together youth (grades 2-5), their families, librarians, and professional engineers in an informal environment centered on engaging youth with age-appropriate, technology-rich STEM learning experiences fundamental to the engineering design process. The overarching aim is to better understand how youth's learning preferences or dispositions relate to their STEM learning experiences. It also seeks to build community members' capacity to inspire and educate youth about STEM careers. The project team includes the Space Science Institute's (SSI) National Center for Interactive Learning (NCIL), the University of Virginia (UVA) and the American Society of Civil Engineers (ASCE). This team builds on the scope and reach of a prior NSF-funded project called the STAR Library Education Network (STAR_Net). As an extension of this prior work, Project BUILD will collaborate with 6 public libraries (3 urban and 3 rural) and their local ASCE Branches. Two libraries have been selected to serve as pilots: High Plains Public Library in Colorado and the African-American Research Library and Cultural Center in Florida. All partner libraries will develop a plan for recruiting participants from groups currently underrepresented in STEM professions. Project BUILD's specific aims are to 1) Engage underserved audiences, 2) Build the capacity of participating librarians and ASCE volunteers, 3) Increase interest and engagement in STEM activities for youth in grades 2-5 and their families, and 4) Conduct a comprehensive education research project. Program components include the following: 1) Community Dialogue Events, 2) a Professional Development Program for partner librarians and ASCE volunteers, and 3) Development of a Technology-rich Programming Kit and Circulating STEM Kit program. Two research questions will be addressed: 1) What common factors might identify youth who engage in project activities and what factors might differentiate between youth who continue with program engagement and those who do not? and 2) What programmatic factors (i.e. design and composition of program activities, library recruitment, librarian engagement, professional engineer engagement, etc.) might influence youth's initial and continued engagement in project activities as well as youth's reported future career interests? An external evaluation will investigate the quality of the project's process as well as its impact and effectiveness. Benefits to the participating libraries' communities, library and engineering professionals, and the education community will be achieved through 1) Community Dialogue events; 2) Library and Librarian Outreach; 3) ASCE Outreach; and 4) Publication of Research and Evaluation results.
This project develops and examines place-based learning using mobile augmented reality experiences for rural families where museums and science centers are scarce yet where natural resources are rich with outdoor trails, parks, and forestlands. The collaborative research team, with members from rural libraries, outdoor learning centers, learning scientists at Penn State University, and rural communities in Pennsylvania, will develop augmented reality and mobile learning resources for families and children aged from 4 to 12. The goal is to help people see what is not visible in real-time in order to learn about life and earth sciences based on local watersheds, trees, and seasonal cycles that are familiar and relevant to rural communities. To accomplish this goal, the project team will create scientifically meaningful experiences for rural families and children in their out-of-school time through three iterations of research and design. Although there is evidence that augmented reality can support learning, little empirical research has been conducted to determine what makes one type of augmented learning experience more effective than others in outdoor learning spaces. This project will produce research findings on the utility of augmented reality for science learning with families and youths outdoors. This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants
Through a four-year design-based research study, researchers will investigate three research questions. (1) How can outdoor learning experiences be enhanced with augmented reality and digital resources in ways that make science more visible and interesting?; (2) How do different forms of augmentations on trails and in gardens support science learning? 3) What social roles do children and parents play in supporting each other's science learning and connections to rural communities? Data collection includes video-recordings of children and families in the outdoors, learning analytics of people's behavior, and interviews with rural families. The project's research design will allow for the development of theory, which supports rural families learning science within and about their communities. At the end of the project, the team will offer generalizable design principles for technologically-enhanced informal learning for outdoor displays, gardens, and trails.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. This project will develop a national infrastructure of state and regional partnerships to scale up The Franklin Institute's proven model of Leap into Science, an outreach program that builds the capacity of children (ages 3-10) and families from underserved communities to participate in science where they live. Leap into Science combines children's science-themed books with hands-on science activities to promote life-long interest and knowledge of science, and does so through partnerships with informal educators at libraries, museums, and other out-of-school time providers. Already field-tested and implemented in 12 cities, Leap into Science will be expanded to 90 new rural and urban communities in 15 states, and it is estimated that this expansion will reach more than 500,000 children and adults as well as 2,700 informal educators over four years. The inclusion of marginalized rural communities will provide new opportunities to evaluate and adapt the program to the unique assets and needs of rural families and communities.
The project will include evaluation and learning research activities. Evaluation will focus on: 1) the formative issues that may arise and modifications that may enhance implementation; and 2) the overall effectiveness and impact of the Leap into Science program as it is scaled across more sites and partners. Learning research will be used to investigate questions organized around how family science interest emerges and develops among 36 participating families across six sites (3 rural, 3 urban). Qualitative methods, including data synthesis and cross-case analysis using constant comparison, will be used to develop multiple case studies that provide insights into the processes and outcomes of interest development as families engage with Leap into Science and a conceptual framework that guides future research. This project involves a partnership between The Franklin Institute (Philadelphia, PA), the National Girls Collaborative Project (Seattle, WA), Education Development Center (Waltham, MA), and the Institute for Learning Innovation (Corvallis, OR).