As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Innovations in Development project will develop and test a model for audience assessment that STEM professionals can use during their public engagement efforts. These on-the-spot assessments will allow scientists to monitor audience understanding and use this to make immediate improvements to their activity. This project fills a critical gap in the field of public science communication as expressed by outreach scientists and by the professional and academic organizations that train them. Project partners include The Astronomical Society of the Pacific, Oregon State University, Pacific Science Center, and the National Radio Astronomy Observatory. Research questions include: (1) How and under what conditions can scientists, regardless of discipline, learn to build assessment into outreach activities? (2) To what degree are scientists willing and able to change their outreach activities to include assessment to ascertain audience attentiveness or understanding? (3) To what degree will scientists be able to adjust their outreach and engagement efforts based on audience feedback, and what support do they need to do so? (4) With constraints on their time and resources, how can the model help scientists conduct audience assessment on their own? (5) Are audience members finding events more enjoyable and fruitful that use assessment techniques? Several iterative rounds of R&D will be used in the spiral design-based approach. A group of Design Testers will start each of the iterative cycles for the initial development and testing techniques, associated training, and support. Later phases will use Field Testers using a rubric for usability, validity, and reliability of various strategies. The summative evaluation of the project will begin early and add data over the life of the project. A mixed methods approach will be used including surveys of scientists once each year after they have participated in the Prototype training. Descriptive and inferential statistics will be used to document the extent to which the project has achieved its intended outcomes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This workshop is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project will conduct a two-day workshop that will gather citizen science project leaders to address barriers in citizen science research and infrastructure: The inability to holistically study the movement, engagement, persistence and learning outcomes among volunteers engaged in multiple projects. The past few years have been a time of tremendous growth in awareness of and interest in citizen science projects. The project will address an increasing gap preventing projects in three now-popular categories (apps, projects hosted on government websites, and event-based projects) from adopting the digital tools created and available through SciStarter.com. The workshop will bring citizen science project leaders together to deepen an understanding of their needs regarding the adoption of digital tools, developed by Scistarter, which will result in more comprehensive data in support of research in informal science learning outcomes of volunteers engaged in citizen science across projects and platforms. The in-person and online contributions from participants will guide the development of resources and tutorials to scale adoption.
SciStarter is a repository of hundreds of citizen science projects. Through previous NSF support, SciStarter developed digital affiliate tools which project leaders use on their own websites to enable analytics (statistics gathered from user activity online) to help projects more easily recruit and coordinator volunteers, help volunteers track their contributions across projects and platforms, and help researchers holistically study the movement and learning outcomes across projects and platforms. The proposed workshop will facilitate iteration and adoption of the tools among three classes of projects, not originally accounted for, which have dramatically increased in numbers during the past year: 1) app-based projects, 2) projects hosted on government websites, and 3) event-based projects.. By co-designing and implementing iterative versions of the tools among these projects, the project will address important gaps in research, enable a richer, more comprehensive understanding of volunteer engagement patterns, and discover opportunities to build a stronger community of citizen science practitioners who collaborate to enhance volunteer learning communities. The project will culminate in improved research in this field and improved management of citizen science projects for appropriate recruitment and retention that fosters STEM learning.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. This AISL project investigates how informal programs can broaden participation by building social capital in STEM for youth from underrepresented groups. The project integrates social network analysis with research on informal learning, and draws on a framework to connect learning across a variety of sectors. It builds on evidence that sponsorship of youth interest, affinity-based mentorship, and brokering connections to other settings and opportunities can build social capital and support interest and persistence in STEM. It represents a strategic and timely investment into research that solidifies these emerging insights from research and practice, conducting focused investigation into relational supports for STEM interests that are particularly well suited to informal programs.
The project is guided by two research questions: (1) What forms of social capital are tied to persistence in and connecting across informal STEM programs for youth from underrepresented groups? (2) What program features--specifically sponsorship, mentorship, and brokering--grow these social supports for persistence in and connecting across informal STEM programs for underserved youth? These questions are addressed through a mixed methods 18-month cross-sectional study of 200 students in three informal programs in Orange County, California that offer project-based engineering and coding programs, support mentorship, and focus on groups underrepresented in STEM. The sample will include three age categories, capturing the transition to high school, persistence during high school, and transition to college and career. Teens will be interviewed three times at 6-month intervals, spanning these transitions. The goal of this research and effort is to determine if social capital plays an extra ordinary role in learning by this group.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This project develops and examines place-based learning using mobile augmented reality experiences for rural families where museums and science centers are scarce yet where natural resources are rich with outdoor trails, parks, and forestlands. The collaborative research team, with members from rural libraries, outdoor learning centers, learning scientists at Penn State University, and rural communities in Pennsylvania, will develop augmented reality and mobile learning resources for families and children aged from 4 to 12. The goal is to help people see what is not visible in real-time in order to learn about life and earth sciences based on local watersheds, trees, and seasonal cycles that are familiar and relevant to rural communities. To accomplish this goal, the project team will create scientifically meaningful experiences for rural families and children in their out-of-school time through three iterations of research and design. Although there is evidence that augmented reality can support learning, little empirical research has been conducted to determine what makes one type of augmented learning experience more effective than others in outdoor learning spaces. This project will produce research findings on the utility of augmented reality for science learning with families and youths outdoors. This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants
Through a four-year design-based research study, researchers will investigate three research questions. (1) How can outdoor learning experiences be enhanced with augmented reality and digital resources in ways that make science more visible and interesting?; (2) How do different forms of augmentations on trails and in gardens support science learning? 3) What social roles do children and parents play in supporting each other's science learning and connections to rural communities? Data collection includes video-recordings of children and families in the outdoors, learning analytics of people's behavior, and interviews with rural families. The project's research design will allow for the development of theory, which supports rural families learning science within and about their communities. At the end of the project, the team will offer generalizable design principles for technologically-enhanced informal learning for outdoor displays, gardens, and trails.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. This project examines the conditions in which families and young learners most benefit from "doing science and math" together among a population that is typically underserved with respect to STEM experiences--families experiencing poverty. This project builds on an existing program called Teaching Together that uses interactive parent-child workshops led by a museum educator and focused on supporting STEM learning at home. The goal of these workshops is to increase parents'/caregivers' self-perception and ability to serve as their child's first teacher by supporting learning and inquiry conversations during daily routines and informal STEM activities. Families attend a series of afternoon and evening workshops at their child's preschool center and at a local children's museum. Parents/Caregivers may participate in online home learning activities and museum experiences. The project uses an experimental design to test the added value of providing incremental supports for informal STEM learning. The study uses an experimental design to address potential barriers parents/caregivers may perceive to doing informal STEM activities with their child. The project also explores how the quantity and quality parent-child informal learning interactions may relate to changes in children's science and mathematics knowledge during the pre-kindergarten year. The project partners include the Children's Learning Institute at the University of Texas Health Science Center at Houston and the Children's Museum of Houston.
The project is designed to increase understanding of how parents/caregivers can be encouraged to support informal STEM learning by experimentally manipulating key aspects of the broader expectancy-value-cost motivation theory, which is well established in psychology and education literatures but has not been applied to preschool parent-child informal STEM learning. More specifically, the intervention conditions are designed to identify how specific parent supports can mitigate potential barriers that families experiencing poverty face. These intervention conditions include: modeling of informal STEM learning during workshops to address skills and knowledge barriers; materials to address difficulties accessing science and math resources; and incentives as a way to address parental time pressures and/or costs and thereby improve involvement in informal learning activities. Intervention effects will be calculated in terms of effect sizes and potential mediators of change will be explored with structural equation modeling. The first phase of the project uses an iterative process to refine the curriculum and expand the collection of resources designed for families of 3- to 5-year-olds. The second phase uses an experimental study of the STEM program to examine conditions that maximize participation and effectiveness of family learning programs. In all, 360 families will be randomly assigned to four conditions: 1) business-as-usual control; 2) the Teaching Together core workshop-based program; 3) Teaching Together workshops + provision of inquiry-based STEM activity kits for the home; and 4) Teaching Together workshop + activity kits + provision of monetary incentives for parents/caregivers when they document informal STEM learning experiences with their child. The interventions will occur in English and Spanish. A cost analysis across the interventions will also be conducted. This study uses quantitative and qualitative approaches. Data sources include parent surveys and interviews, conversation analysis of home learning activities, parent photo documentation of informal learning activities, and standardized assessments of children's growth in mathematics, science, and vocabulary knowledge.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. This Pilots and Feasibility Studies project will design and study computer science education programs in three New York State prisons and cutting-edge computer science learning spaces in New York City and Bennington, VT for recently released men and women. These two sites of investment will be designed and operated with the goals of bridging informal STEM education inside the prison and post-release. The project aims to help incarcerated and formerly incarcerated men and women gain hands-on experience and technical fluency in computer science. The project will also help people build effective pathways that link training and informal learning communities in prison to professional and educational success after prison.
The research will examine how informal STEM learning can 1) provide incarcerated and post-incarceration populations more effective pathways into computer science and technology professions, and 2) help revitalize neighborhoods struggling with high rates of incarceration and high rates of adults under correctional supervision. Results will be shared among the STEM learning community and prison educators through existing networks, scholarly and journalistic publications, and conferences. As a pilot program, this project aims to develop a comprehensive, rigorous, and transferable model that may be used as a resource in other prison education programs as well as in rural and urban communities across the United States seeking to address economic and educational inequalities of post-incarceration life.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Over the last decade there has been significant growth in the number of afterschool programs that offer science activities to youth. Measuring the quality and contributions of these programs to youth learning is important to both the afterschool organizations and the communities that support them, including participating youth and their families. To address the range and evolving interests and capacities of all young people within a community, there are, by necessity, a wide range of types of afterschool science programming. Such programming may vary by focus (e.g., botany, astronomy, computer science, engineering, or zoology), structure (e.g., hands-on, place-based, on-line, or in partnership with local industry) and other factors. Across the range of programming, there are different intended learning goals and opportunities for students. For these reasons, a range of measurement tools are needed to monitor the quality and outcomes of wide range of afterschool science programs. To explore the current state of evaluation and measurement tools for use in afterschool science programs, the University of Washington, in partnership with the Afterschool Alliance and the National Girls Collaborative, will design and host a conference for afterschool STEM leaders, researchers, and evaluators. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This conference is grounded in the programmatic priorities and strategies of afterschool organizations. The goals of the conference are to (a) articulate and map the range of intended outcomes of afterschool STEM programs; (b) identify how existing measurement and evaluation tools map on to intended outcomes; and (c) identify overlaps, complementarities, and gaps in the available tools in order to provide guidance to (i) practitioners on how and why to select current evaluation tools and (ii) researchers on directions for future tool development. Tangible convening products include:
*A detailed, visual representation ("intended outcomes map") of the range of outcomes afterschool programs are seeking to achieve, related to student learning, educator capacity, program quality, family impacts, learning ecosystems connectivity;
*A taxonomy of current evaluation instruments aligned to these outcomes, with an explanation of how they overlap or differentiate both methodologically and theoretically;
*The identification of the areas where further work is needed, including further specification of learning outcomes and future development of evaluation tools.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Pilot and Feasibility study will investigate strategies for enhancing the mathematics in museum-based making and tinkering activities and lay the foundation for a full research study on broadening family participation in mathematics through making. This proposal builds directly on the NSF-funded Math in the Making convening. During this convening, questions about how to authentically highlight and enhance the mathematics in making and tinkering experiences, and how different math-enhancement approaches might influence learner experiences and outcomes, emerged as critical issues for researchers, educators, and mathematicians alike. The project aims to provide a practical lens to help researchers and educators connect topics across STEM with making and tinkering experiences. The project also seeks to advance theoretical understandings of museum-based learning by exploring ways that activity design and facilitation strategies influence how visitors understand the nature and goals of the experience and, in turn, how these visitor experiences shape learning outcomes. The project is designed to explore the most promising of these math-enhancement strategies in more depth, to propose as a next project and develop a theoretical framework for understanding and describing how these strategies influence how families understand and engage with the mathematics in maker experiences. Through several culturally-responsive approaches developed in collaboration with community-based organizations, the project will research how mathematics in maker experiences influences participant engagement and learning. The project will culminate in the design of a research study. Reports and resources developed by the project will be broadly disseminated to researchers, mathematicians, and educators. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This award was provided as part of NSF's Social, Behavioral and Economic Sciences Postdoctoral Research Fellowships (SPRF) program and is supported by SBE's Developmental Sciences program and the Directorate for Education and Human Resources' (EHR) Advancing Informal STEM Learning program. The goal of the SPRF program is to prepare promising, early career doctoral-level scientists for scientific careers in academia, industry or private sector, and government. SPRF awards involve two years of training under the sponsorship of established scientists and encourage Postdoctoral Fellows to perform independent research. NSF seeks to promote the participation of scientists from all segments of the scientific community, including those from underrepresented groups, in its research programs and activities; the postdoctoral period is considered to be an important level of professional development in attaining this goal. Each Postdoctoral Fellow must address important scientific questions that advance their respective disciplinary fields. Under the sponsorship of Dr. Sandra D. Simpkins at the University of California, Irvine, this postdoctoral fellowship award supports an early career scientist exploring high-quality and culturally responsive, math afterschool program (ASP) practices for under-represented minority (URM) youth. Mathematical proficiency is the foundation of youth's STEM pursuits. Yet today, far too many youth do not pursue STEM based on a perception that they are "not good at math". Students need to engage in contexts that spark their interest and their continued mastery and growth. ASPs are settings for such dynamic opportunities, particularly for URM students such as Latinos who attend lower quality schools and do not feel supported. In college, URM students often struggle with uninspiring and culturally incongruent STEM learning environments. The intergenerational nature of university-based STEM ASPs, whereby younger students are paired with undergraduate (UG) mentors, are opportunities to support both K-12 and UG students' motivational beliefs in math and STEM more broadly. This project will examine these intergenerational developmental processes in the context of a math enrichment ASP located at a Hispanic-Serving Institution. By studying how ASPs can serve as an important lever for promoting URM students' access and success in STEM, this project seeks to meaningfully inform efforts to broaden the participation of underrepresented groups in these fields.
This project seeks to understand how participating in a math enrichment ASP supports both youth participants' and UG mentors' motivational beliefs in math; to describe high-quality and culturally responsive practices; and to understand how to support the effectiveness of youth-staff relationships. To accomplish these research objectives, data will be collected from both youth participants and UG mentors through multiple methods including surveys, in-depth interviews, participant-observations, and video observations of youth-staff interactions. This project will add to our understanding of university-ASP partnerships. Further, the knowledge gained from this study will impact the larger landscape of practice and research on STEM ASPs by 1) addressing critical gaps in the current literature on high-quality and culturally responsive STEM ASP practices and 2) informing ASP staff development training. Overall, this mixed methods project will provide critical and rich information on the ways that ASPs can effectively deliver on its promise of promoting positive development for all youth, especially URM youth who may need and benefit from these spaces the most. The invaluable insight garnered from this study will be disseminated to traditional academic audiences to advance knowledge, as well as to local, state, and national organizations to inform the larger landscape of practice in STEM ASPs.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Explore the Science of Spring: A Live Media Event is an Innovations in Development project produced by the signature PBS series Nature. The new primetime series Spring LIVE (working title) will break the frame of a traditional documentary, letting viewers themselves explore the dramatic seasonal changes of spring through the immediacy of live television. On-camera hosts, scientists and naturalists in locations across the U.S., and scores of citizen scientists will use observation and scientific inquiry to explore the workings of nature during this season of rebirth. The unfolding stories of seasonal change will illuminate larger scientific insights--into the biodiversity of species in habitats, the interconnectedness of plants and animals in diverse ecosystems, the global phenomenon of species migration, and how spring "green-up" can be affected by environmental change--while inspiring appreciation for species conservation and habitat preservation. Spring LIVE is conceived as an ongoing series, with this inaugural season composed of three one-hour programs broadcast live on three consecutive nights, along with real-time interactions via Facebook. Reaching long-standing Nature viewers (2.5 million per episode), Spring LIVE will seek to turn mature adults and diverse families into citizen science doers, and leverage younger Nature online audiences through social media and community engagement in partnership with citizen science projects.
Spring LIVE will build public knowledge of and engagement in phenology and citizen science. The project will also conduct knowledge-building research on the effectiveness of Facebook as a science learning tool. It will experiment with eliciting audience participation via Facebook within the live shows to generate synchronous, second-screen thought and discussion. An exploratory study by Multimedia Research will look at the impact of this feature, addressing the question: To what extent and how does Facebook interactivity within live science shows impact adult engagement, learning and motivation? Spring LIVE will also engage multiple partners to expand reach and impact and build capacity in their fields. National partners include the National Park Service and Next Avenue; citizen science partners include Celebrate Urban Birds, National Phenology Network, Monarch Blitz, and SciStarter, among others. PBS stations will work with these organizations to involve diverse, intergenerational audiences in observation of nature and seasonal change. Project evaluation, implemented by Knight Williams Research Communications, will focus on the impact of live television on science learning, and the success of the integration of citizen science projects on air, online, and in communities. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Exploratorium in San Francisco, in collaboration with social science researchers at Arizona State University and Rutgers University, will conduct a two-year Pilots and Feasibility Studies project that will test a new model for integrating museum exhibits with public participation in scientific research, called Exhibit-Based PPSR (E-PPSR). The team will create a mini-exhibition about social psychology that (a) engages science museum visitors in investigating and reflecting on social factors that promote cooperation or not, (b) builds public awareness of the science of social psychology and (c) generates valid data for academic research in this area.
The Exploratorium will build on its long history of creating participatory exhibits that engage visiting groups in social science learning and in self-reflective metacognition. This expands beyond the typical lab-in-the-museum in which individual visitors interact with researchers' laptops or simple props. The model takes the exhibit experiences further, injecting the dimension of public participation in social psychology research. By voluntarily and anonymously contributing demographic and response data, large numbers of people from mixed-age groups will help social scientists gain new insights into cooperation among a broader sample of humanity than are usually studied in university labs. The E-PPSR lab is always available, rather than open only when a researcher is present. The model also incorporates research on learning and assesses the effect of E-PPSR on social science learning experiences. Do museum visitors build greater conceptual understanding of the social science by contributing to real research and seeing their own responses within the larger dataset? Do they attend more deeply to debriefing activities when they have contributed their own data? The three main deliverables include: 1) a prototype Exhibit-based PPSR laboratory at the Exploratorium comprised of one exhibit for gaining informed consent, three 'Data-Catcher' exhibits modified to record anonymous responses when visitors opt-in to contributing to social psychological research, and one debriefing exhibit. A back-end database will send data to the academic researchers; 2) evaluation studies that test the E-PPSR model. The studies will assess the success of debriefing approaches, the effectiveness of recruitment and the impact of E-PPSR on learning. The team will publish a journal article describing the E-PPSR model and academic research findings. The team will also organize a conference session with others in the museum field who manage in-house academic research laboratories; and 3) a report by the academic partners describing the impact of the project on their research program.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Informal STEM education needs high quality program evaluation. Research is particularly needed on the relationship between STEM outcomes and positive youth development (PYD) / socio-emotional learning (SEL). This pilot and feasibility project involves a collaboration between experts in STEM education, out-of-school time programs (OST), PYD, SEL, evaluation, and program development. STEM Scouts helps youth grow in character and develop skills using experiential activities and interaction with STEM professionals. This project expands the implementation of the Systems Evaluation Protocol (SEP), an evidence-based approach to evaluation planning.
The goals are to: 1) develop a theory of change for STEM Scouts highlighting the relationship between STEM outcomes and PYD/SEL outcomes; 2) pilot enhancements to the SEP (System Mapping, Ecosystem Modeling, and Model Validation); and, 3) determine the feasibility of conducting a national STEM Scouts study. STEM Scouts leadership and project researchers will work through the SEP to generate a stakeholder map, logic model, and pathway model (PM). Five STEM Scouts Labs across the country will participate in focus groups where they will complete SM to identify the system in which the Lab exists (e.g., stakeholders and decision-makers), review and revise their system maps, identify key outcomes and connections in the PM, and discuss how the PM reflects their STEM Scouts experiences. It is hypothesized the enhanced SEP will enable the working group to better understand factors hindering or enabling program and evaluation feasibility and success. Findings will be disseminated to the evaluation/research community, OST program providers/developers and the public.
This project is funded by the National Science Foundation's Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.