In April 2018, FHI 360, under the leadership of Maryann Stimmer and Merle Froschl, convened a meeting of thought leaders in Washington, D.C. to capture a “snapshot” of STEM education. They subsequently conducted additional interviews with more than 50 local and national policy leaders; public and private funders; researchers; PreK-12 and post-secondary educators; parents, and leaders of afterschool programs, science centers and youth-serving organizations. The purpose of this summary report is to identify current trends and gaps to inform research, policy, and practice in order to reinforce
With funding from a Smithsonian Institution Youth Access Grant the National Museum of Natural History offered a program pipeline that would engage underserved Washington D.C. students in programming designed to encourage academic and career STEM interest. Beginning with collaboration with community educators, the pipeline funneled students into a Teen Night Out Science Night to interest them in science workshop series offered after school and on weekend and ultimately into volunteer and internship opportunities. This report provides detailed view of methods, analyses, results, and conclusions
Broadening participation efforts need to move beyond the programmatic to the institutional. Embedding inclusion throughout an organization’s operations will lead to more comprehensive, better supported, and more impactful and sustainable results.
About this resource:
This is a practice brief produced by CAISE's Broadening Participation in STEM Task Force to help informal STEM education (ISE) and science communication groups reflect on and strengthen their efforts to broaden participation in STEM. It is part of a larger professional development toolkit, developed for those who lead staff
Informal STEM education institutions seek to engage broader cross sections of their communities to address inequities in STEM participation and remain relevant in a multicultural society. In this chapter, we advance the role that evaluation can play in helping the field adopt more inclusive practices and achieve greater equity than at present through evaluation that addresses sociopolitical contexts and reflects the perspectives and values of non-dominant communities. To do this for specific projects, we argue that evaluation should privilege the voices and lived experiences of non-dominant
This project will advance efforts of the Innovative Technology Experiences for Students and Teachers (ITEST) program to better understand and promote practices that increase student motivations and capacities to pursue careers in fields of science, technology, engineering, or mathematics (STEM) by designing, implementing, and testing an afterschool internship program that will engage older youth in work-based learning experiences in in STEM fields. The new model program will link the resources and learning approaches of the Global Learning and Observations to Benefit the Environment (GLOBE) program to career academies where youth from populations underrepresented in STEM fields will gain direct experiences in data collection and analysis through student-led investigations in the geosciences and environmental studies. Two key outcomes of this project will be: (a) Development of a replicable model of an afterschool STEM internship program for informal STEM learning environments and schools across the nation, and (b) Development of a set of measurement tools and approaches that can assess and promote understanding regarding how youth think and feel about science and their possible future roles in science careers. Participating youth will master scientific practices and become immersed in science culture through opportunities to develop research projects, interact with scientists, and collaborate with fellow student-researchers. In the process, they will develop collaboration and communication skills, and gain an increased sense of identity and agency in science fields. They will also learn new strategies to attain their career goals.
In developing and testing the new model of an afterschool program focusing on STEM careers, the project will draw on both existing and emerging knowledge from three areas of inquiry: informal STEM learning, youth development, and work-based learning. The project will bring together theory related to work-based learning and apprenticeship to knowledge about informal STEM learning and youth development, addressing the needs of older youth as they transition to adulthood. The program will also explore the use of measurement tools that address workforce-related student learning goals in addition to social-emotional learning and STEM learning goals, adapting existing tools and developing new tools as needed. The result will be a replicable model for an afterschool, career-focused internship that facilitates STEM learning and identity, employing youth development principles, such as experiential learning, peer collaboration, adult mentoring, and meaningful contributions to the world beyond school. The project will use a mixed-methods approach to investigate four research questions: (1) What aspects of the program are most important for promoting the development of scientific practices, socio-emotional learning, and career skills? (2) How can afterschool informal science learning be designed to address the perceptions and needs of diverse groups, especially those from populations underrepresented in STEM? (3) How do youth make gains in developing facility with STEM practices, key social-emotional outcomes needed in work and civic life, and career development knowledge? And (4) How do we accurately measure development of scientific practices, socio-emotional learning and career skills? The project will develop pretest and posttest self-report measures to gauge program influence on social-emotional outcomes and career-related outcomes, and performance-based assessments and rubrics will be used to assess culminating science projects. Other factors contributing to the success of the new model will be examined through analysis of coach instructional logs, surveys, and questions, as well as participant observations, interviews, and focus groups. Project participants will be youth of ages 14-18 recruited from ten inner-city schools having large populations of students from groups underrepresented in STEM fields. Participants will meet in teams of approximately 14 interns for a total of 2.5 hours per week for 32 weeks. Each team will also meet an additional 4-6 times for weekend or overnight outings associated with their study sites.
DATE:
-
TEAM MEMBERS:
Manuel AlonsoCathy RingstaffSvetlana Darche
STEM out-of-school time (OST) programs play an important role in helping youth develop the 21st century skills they need to prepare them for the workforce, particularly the teamwork skills necessary for the growing collaborative nature of work in STEM (National Research Council, 2015). However, there is a lack of appropriate tools to evaluate this key programmatic outcome in STEM OST settings. Through funding from the National Science Foundation, we carried out the Collaboration in the 21st Century (C2C) project to help address this need by developing and validating a survey, the Youth
Twenty-first century skills are vital for preparing youth for careers in science, technology, engineering, and math (STEM) fields. STEM out-of-school time (OST) programs play an important role in helping youth develop these skills, particularly the teamwork skills necessary for the growing collaborative nature of STEM jobs. However, there is a lack of appropriate measures to evaluate this key programmatic outcome in STEM OST settings. This dissertation research addresses the lack of measures through the development of an instrument to assess team communication skills in middle and high school
Over the last decade there has been significant growth in the number of afterschool programs that offer science activities to youth. Measuring the quality and contributions of these programs to youth learning is important to both the afterschool organizations and the communities that support them, including participating youth and their families. To address the range and evolving interests and capacities of all young people within a community, there are, by necessity, a wide range of types of afterschool science programming. Such programming may vary by focus (e.g., botany, astronomy, computer science, engineering, or zoology), structure (e.g., hands-on, place-based, on-line, or in partnership with local industry) and other factors. Across the range of programming, there are different intended learning goals and opportunities for students. For these reasons, a range of measurement tools are needed to monitor the quality and outcomes of wide range of afterschool science programs. To explore the current state of evaluation and measurement tools for use in afterschool science programs, the University of Washington, in partnership with the Afterschool Alliance and the National Girls Collaborative, will design and host a conference for afterschool STEM leaders, researchers, and evaluators. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This conference is grounded in the programmatic priorities and strategies of afterschool organizations. The goals of the conference are to (a) articulate and map the range of intended outcomes of afterschool STEM programs; (b) identify how existing measurement and evaluation tools map on to intended outcomes; and (c) identify overlaps, complementarities, and gaps in the available tools in order to provide guidance to (i) practitioners on how and why to select current evaluation tools and (ii) researchers on directions for future tool development. Tangible convening products include:
*A detailed, visual representation ("intended outcomes map") of the range of outcomes afterschool programs are seeking to achieve, related to student learning, educator capacity, program quality, family impacts, learning ecosystems connectivity;
*A taxonomy of current evaluation instruments aligned to these outcomes, with an explanation of how they overlap or differentiate both methodologically and theoretically;
*The identification of the areas where further work is needed, including further specification of learning outcomes and future development of evaluation tools.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Informal STEM education needs high quality program evaluation. Research is particularly needed on the relationship between STEM outcomes and positive youth development (PYD) / socio-emotional learning (SEL). This pilot and feasibility project involves a collaboration between experts in STEM education, out-of-school time programs (OST), PYD, SEL, evaluation, and program development. STEM Scouts helps youth grow in character and develop skills using experiential activities and interaction with STEM professionals. This project expands the implementation of the Systems Evaluation Protocol (SEP), an evidence-based approach to evaluation planning.
The goals are to: 1) develop a theory of change for STEM Scouts highlighting the relationship between STEM outcomes and PYD/SEL outcomes; 2) pilot enhancements to the SEP (System Mapping, Ecosystem Modeling, and Model Validation); and, 3) determine the feasibility of conducting a national STEM Scouts study. STEM Scouts leadership and project researchers will work through the SEP to generate a stakeholder map, logic model, and pathway model (PM). Five STEM Scouts Labs across the country will participate in focus groups where they will complete SM to identify the system in which the Lab exists (e.g., stakeholders and decision-makers), review and revise their system maps, identify key outcomes and connections in the PM, and discuss how the PM reflects their STEM Scouts experiences. It is hypothesized the enhanced SEP will enable the working group to better understand factors hindering or enabling program and evaluation feasibility and success. Findings will be disseminated to the evaluation/research community, OST program providers/developers and the public.
This project is funded by the National Science Foundation's Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Techbridge Girls’ mission is to help girls discover a passion for science, engineering, and technology (SET). In August 2013, Techbridge Girls was awarded a five-year National Science Foundation grant to scale up its afterschool program from the San Francisco Bay Area to multiple new locations around the United States. Techbridge Girls began offering afterschool programming at elementary and middle schools in Greater Seattle in 2014, and in Washington, DC in 2015.
Education Development Center is conducting the formative and summative evaluation of the project. To assess the implementation
Techbridge Girls’ mission is to help girls discover a passion for science, engineering, and technology (SET). In August 2013, Techbridge Girls was awarded a five-year National Science Foundation grant to scale up its afterschool program from the San Francisco Bay Area to multiple new locations around the United States. Techbridge Girls began offering afterschool programming at elementary and middle schools in Greater Seattle in 2014, and in Washington, DC in 2015.
Education Development Center is conducting the formative and summative evaluation of the project. To assess the implementation