The summative evaluation documents and articulates what SCIENCES has improved or changed, and in what ways. The final design of the summative evaluation was based on findings from the front-end and formative evaluations, including using participatory evaluation techniques to engage community members in discussing their experience with the programs and assessment of community needs and assets at the close of the project.
The goal of the summative evaluation was to address discrete program impacts in the context of the project, as well as the cross-program impact of providing a thematically
In The Nature of Community: SCIENCES, we share the lessons learned from an innovative partnership designed to leverage the strengths of two nonprofit organizations—a large cultural institution and a smaller, deeply-rooted community-based organization, both of which offer informal science education expertise.
You’ll read first-hand reflections of how staff members, community leaders and members, children, and adults experienced this partnership: the expectations, surprises, challenges, successes, and lessons learned. We hope the description of this partnership inspires other organizations to
The Science Fairs Under the 'Scope Study's key findings are summarized here on the topics of:
Models And Elements Of Middle School Science Fairs
Science And Engineering Practices
Cost Of Science Fairs
Parent Involvement In Science Fairs
Science Interest And Identity.
We collected data from middle school science fairs held in schools across the country to understand:
What are the basic models and elements of middle school science fairs;
If and how science fairs increase students’ interest in science, technology, engineering or math (STEM) and/or STEM careers
If and how participation in select models of middle school science fairs enhance students’ mastery of the science and engineering practices; and
What costs and resources are required to implement an effective middle school science fair?
In this article we explore how activity design and learning contexts can influence youth failure mindsets through a case study of five youth who described failure as sometimes a good thing and sometimes a bad thing (a perspective we characterize as Failure as Mosaic, described in the article). These youth and their descriptions of failure-positive and failure-negative experiences offer a unique opportunity to identify how experiences can be designed to support learning and persistence. In order to understand differing views of failure among youth, we researched the following questions:
Seeking to use narrative as a vehicle for getting more young people interested in STEM, the National Science Foundation supported a WETA/PBS NewsHour initiative to adapt their Student Reporting Labs (SRL) curriculum to feature a focus on STEM content in 2015. NewsHour selected Knology to run a four-year evaluation of the project from 2015 to 2019. Building on earlier research about student-led science journalism (e.g. Polman & Hope, 2014; Nicholas, 2017), this project suggests that having students develop and produce narratives about complex STEM topics may make these topics far less
This broadening participation project will focus on a regional workshop aimed at increasing Historically Black Colleges and Universities' (HBCUs) capacity to develop high quality proposals for future competitions of various programs in the Division of Research on Learning. The proposed effort will occur through three specific steps involving a: (1) pre-workshop webinar to introduce and lay the foundation for the opportunity; (2) full two-day workshop to engage participants in a rigorous grant-writing exercise; and (3) post workshop follow-up to provide ongoing support and proposal development guidance. Through a theory-driven process, the goal is to establish some degree of conformity for maximizing grant productivity around strategies and ideas shown to be effective in retaining students in the STEM pipeline.
The multi-tiered workshop will establish a launching pad for increasing attendees' capacity to build on prior knowledge and use best practices to improve future grant writing efforts. Specialized activities will help prepare HBCUs to increase their contributions to diversifying the future STEM workforce, support innovation and creativity in STEM fields, expand networking strategies, and promote opportunities to learn. Central to this capacity-building effort will be a focus on understanding the current research context and expectations for competitive participation in funding opportunities offered by NSF. This, in turn, will align with the Foundation's strategic direction for broadening participation in STEM through meaningful cutting-edge STEM education research. Resources from the workshop will be made available online to facilitate broader dissemination of information beneficial to HBCUs and other education institutions engaging in broadening participation efforts.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This project will examine the characteristics and outcomes of a large sample of environmental education field trip programs for youth to elucidate program characteristics that most powerfully influence 21st century learning outcomes. Environmental education programs for youth, particularly day-long school trip programs, are popular and reside at the intersection of formal and informal STEM education. Such field trips provide opportunities for diverse audiences to participate in shared learning experiences, but current understanding of what leads to success in these programs is limited. This large-scale study will address this gap in knowledge by investigating the linkages between program characteristics and participant outcomes for at least 800 single-day environmental education field trip programs for youth in grades 5-8, particularly programs for diverse and underserved audiences. This study will result in the identification of evidence-based practices that will inform future program design for a wide variety of settings, including nature centers, national parks, zoos, museums, aquaria, and other locations providing informal environmental education programs.
This Research in Service to Practice study is guided by two research questions: 1) What program characteristics (context, design, and delivery) most powerfully influence learner self-determination and learner outcomes? And 2) Do the most influential program characteristics differ across diverse and underserved audiences (e.g. African American, Hispanic/Latino, economically disadvantaged) and contexts (e.g. rural versus urban)? This project will examine a wide range of program-related factors, including pedagogical approaches and contextual characteristics. A valid and reliable protocol for observing 78 program characteristics hypothesized to influence learner outcomes developed by a previous project will be used to systematically sample and observe 500 single-day environmental education field trip programs for youth in grades 5-8 distributed across at least 40 U.S. states and territories. Programs for diverse and underserved youth will be emphasized, and a diverse set of programs in terms of program type and context will be sought. Data from this sample will be combined with those of an existing sample of 334 programs provided by over 90 providers. The final combined sample of over 800 programs will provide sufficient statistical power to confidently identify which program components are most consistently linked with learning outcomes. This sample size will also enable stratification of the sample for examination of these relationships within relevant subpopulations. Principal component analyses will be used to reduce data in theoretically meaningful and statistically valid ways, and multilevel structural equation modeling will be employed to examine the influences of both participants' individual characteristics and program and context characteristics on participant outcomes. Since one research question focuses on whether program outcomes are the same across different audiences, the project will include at least 200 programs for each of three specific audiences to ensure sufficient statistical power for confidence in the results: primarily African American, primarily Hispanic/Latino, and primarily White.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Robert PowellMarc SternBrandon Frensley
Recognizing that race can influence African American youths' perception of which academic disciplines and careers are available to them, this pilot study will explore how African American youths' physical and social communities can be leveraged to support the evolution of their STEM identity and their ability to recognize their potential as scientists. Unfortunately, many of these youths live in communities that are void of critical resources that research has demonstrated time and time again are critical for success in STEM disciplines and careers. This lived reality for many African American youth is the direct result of long-standing disparities in access and opportunities, fueled by racial socialization and biased institutional structures. This pilot will empower youth to recognize these disparities and use science to provide solutions. One perilous societal disparity experienced in many predominately African American communities is the lack of access to fresh produce and healthy food. As a mechanism for potential resolution, this project will consider the utility of community gardens to address this important community need and as a strategy to engage youth in STEM content and skill development. While this notion is not novel to NSF, the intent to utilize an augmented reality (AR) storytelling platform for data collection and project experiences is innovative. This technology will also provide a space for participants to share their work with each other and their broader communities. To our knowledge, this pioneering approach has not been previously piloted in this context. In addition, the pilot will engage multiple youth serving community-based organizations such as park and recreation centers and faith-based organizations in this work, which is also innovative. This is significant, as youth serving community-based organizations are often play important role in the social, educational, and cultural lives of youth and their families in communities. These organizations are often at the heart of the community, figuratively and literally. If successful, this pilot could be transformative and provide a strong basis to support similar work in other communities.
Over the two-year project duration, eighty African American youth ages 11 -14 will participate in the year-long program, across three youth-serving, community-based organizations at four sites. They will be exposed to relevant agricultural, geological, engineering and technological content through a newly developed curriculum called "Cultivating My Curriculum." Community mentors and undergraduate role models will facilitate the instruction and hands-on experiences in the garden and with the AR platform. A capstone event will be a held for the participants and community to convene to learn more about the results of the pilot and share recommendations with community leaders for improving the disparities identified during the pilot. The research component will focus on: (a) the impact of the sociocultural theoretical framework grounding the work on youths' STEM identities, (b) the integration of the AR tool, and (c) mentorship. Formative and summative evaluation will take place through focus groups, surveys, journals, and youth storytelling. Ultimately, the project endeavors to advance the narrative that African Americans are scientists and that science can be used to improve the lives of African Americans and other groups challenged by structural and racial disparities.
This pilot study is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Harrison PinckneyDavid BoyerBarry GarstDilrukshi Thavarajah
Many of the Hispanic children and families who live in the Rio Grande Valley lack opportunities to engage in inspirational and educational experiences introducing Science, Technology, Engineering and Mathematics (STEM) concepts and related careers. The University of Texas, Rio Grande Valley (UTRGV) will adapt and research the "Energy and U Show," which will introduce thousands of children and families to an exciting and dramatic that shows interconverting different forms of energy. The show will meld the excitement of chemical demonstrations and the natural connection between energy and STEM education in a fully produced, on-stage science extravaganza. A foundational philosophy of the show is that there is additional real value in getting children and youth onto a college campus. For many of its participants, this is their first time sitting in a seat at a university, the first opportunity for them to envision themselves in this environment. In partnership with the University of Minnesota, which originally developed the show, UTRGV will adapt the show, now presented in English, to a bilingual, culturally accessible format that is designed to Hispanic family audiences and student groups in learning about energy and related careers. Evaluation results demonstrate that the show has effectively engaged thousands of Minnesota students. The target audience will be upper elementary (4th-5th grade), middle school students, and their parents. This project will be led by UTRGV, nation's second-largest Hispanic Serving Institution, with a student enrollment of 28,000, of which over 90% are Hispanic and more than 60% are first-generation college students). In addition to the show, the project will include: (1) a manual to guide implementation of the program and related resources at different national or international venues; (2) educational resources for parents, teachers and school counselors introducing STEM careers and specific STEM college majors; (3) mentoring of UTRGV faculty in outreach activities; and (4) dissemination of the show to other campuses and venues.
The project will conduct ongoing research and evaluation guiding the adaptation of the show and investigation of factors contributing to positive educational impacts of the project, which will be carried out by a bilingual/bicultural researcher. Project research instruments will measure student level of engagement, interest and learning, as well as college interest, in surveys and analysis of data pre and post demonstration. The project will specifically investigate the impact of language on student impacts. Each component of this project will be studied to determine program intervention effectiveness (the scientific demonstration and language of the demonstration). To determine program effectiveness, a baseline of data before program implementation will be established concerning Hispanic students, their persistence, and perceptions of the environment. The project will measure parent perceptions of STEM careers for their children through pre and post demonstration surveys and focus groups. Student and parent research participants will be able to use surveys or respond to other research activities in the language of their choice. Project findings will contribute to the knowledge base concerning how linguistically and culturally adapted science shows and related resources adapted into can have positive impacts regarding the STEM knowledge and careers of students and parents from low-income and Hispanic communities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Karen LozanoArturo FuentesAaron MassariBrian Warren
Mentoring is a widely accepted strategy for supporting positive socioemotional and cognitive development across a variety of sectors including education, workforce development, and the justice system. An estimated 2.5 million volunteer mentors support youth development in the United States each year. However, there is broad concern that practice has outpaced empirical testing, with significant gaps in the research literature on important modifiers of mentoring relationships and their impacts. This is especially true for mentoring youth ages 10-14 in STEM. Studying highly successful programs may be one way to better understand the role of mentoring and moderators of mentoring effectiveness. The Science Club, a community-based STEM mentoring program for middle-grade youth in the Chicago area, will provide multiple sites for a research study to examine three important issues for advancing theory and practice for STEM mentoring. These issues include (1) understanding STEM mentoring for youth in the middle grades, (2) identifying outcomes and motivations for scientist mentors to more fully participate in mentoring programs, and (3) examining a model of middle-school-focused STEM mentoring collaboration.
Through a series of three studies, the team will investigate which elements of the mentoring relationships are associated with the demonstrated STEM identity gains in youth participants. The work will also contribute much-needed data on the impact of STEM mentoring relationships on the mentors themselves. Study 1 is designed as a retrospective study of program alumni, both youth and mentors, about the nature and extent of each their STEM identity shifts during their time in Science Club. A purposeful sample of 160+ youth and 100+ mentor alumni will participate. Study 2 is a prospective study of three consecutive cohorts of active Science Club participants, built on data and findings from Study 1. In Study 2, the team will design and implement a new Identity-Focused Mentoring Observation Instrument specifically aimed at exploring the nature and quality of mentoring relationships and their role in science identity development longitudinally. Three independent cohorts of 40 youth and 20 mentors each will participate. Study 3 is retrospective, examining how participating individuals and organizations perceive and are impacted by mentoring. The three studies employ a mixed methods approach utilizing surveys, observations, individual interviews, and document review.
This proposal will fill critical gaps in the mentoring literature regarding the formative middle school years through novel, empirical research. Building on the current literature and practice, outcomes of the work will inform practice and enhance knowledge-building in the field on both mentoring relationships and the collective impact of university-school-OST partnerships.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
There are several critical reasons to understand and support interest development in early childhood: (a) as a primary motivator of engagement and learning; (b) interest development in preschool predicts important learning outcomes and behaviors in early elementary school; and (c) early childhood interests motivate ongoing interest development. Thus, there is growing recognition that interest is not just important but fundamental to education and learning. Head Start on Engineering (HSE) is a multi-component, bilingual (Spanish/English), family-focused program designed to (1) foster long-term interest in the engineering design process for families with preschool children from low-income backgrounds and (2) support family development and kindergarten readiness goals. The HSE program, co-developed with the Head Start community, provides families with developmentally appropriate, story-based engineering design challenges for the home and then connects these to a system of strategically aligned Informal STEM Education (ISE) experiences and resources. This current project, HSE Systems, builds on a previous HSE Pathways project which (a) established that participating families develop persistent engineering-related interests; (b) highlighted the value that the Head Start community has for the program and partnership; and (c) generated a novel, systems perspective on early childhood interest development. The aim of HSE Systems is to develop and test a model of early childhood STEM engagement and advance knowledge of how the family as a system develops interest in STEM from preschool into kindergarten.
Through the Design Based Implementation Research (DBIR) process, the team will iteratively refine and improve the HSE program and theory of change using ongoing feedback and data from staff, families, and partners. It is also designed to explore program impacts on family interest development over a longer period, as children enter kindergarten. The DBIR work will focus primarily on the program model questions, while the case study research will focus on the family interest questions, with both strands informing each other. The initial work is organized around a series of feedback and design-testing cycles to gather input from families and other stakeholders, update the program components and activities in collaboration with families and staff, and prepare for full implementation. During the next phase, the team will implement the full program model with six Head Start classrooms and track family experiences and interest development into kindergarten. During final implementation phase, the team will finish data collection, conduct retrospective analysis with all the data, and update the program model and theory of change.
This project will directly address the AISL program goals by broadening access to early childhood informal STEM education for low-income communities, with a focus on Spanish-speaking families, and building long-term skills and learning dispositions to support STEM learning inside and outside of school. Beyond the topic of engineering, HSE supports Head Start school readiness and child and family development goals, which are the foundation of lifelong success.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.