The SciGirls in Space: Exploring the Moon, Mars and NASA Careers Implementation Evaluation Report focuses on the STEM outreach component of the project for 15 girl-serving organizations who used research-based gender equitable and culturally responsive instructional strategies,NASA-aligned media and STEM activities with a focus on integrating of NASA women role models.
With funding from the National Science Foundation, Secrets of the Universe (SOTU) is a multimedia project that has at its core a 40-minute 3D giant screen film directed by a collaboration of K2 Communications Inc., The Stephen Low Company, and University of California Davis Department of Physics. To support and extend the impact of the film for these audiences, the project features an educational website with a variety of resources, including online videos of science content and characters from the film.
The independent evaluation team from Knight Williams Inc. conducted a summative
Despite decades of policies and programs meant to increase the representation of girls and women in science, technology, engineering, and mathematics (STEM), girls and women of color still represent a much smaller percent of the STEM workforce than they do in the US population. This lack of representation is preventing the US STEM workforce from reaching its true potential. Intersecting inequalities of gender, race, ethnicity, and class, along with stereotypes associated with who is successful in STEM (i.e., White men), lead to perceptions that they do not belong and may not succeed in STEM. Ultimately, these issues hinder girls’ STEM identity development (i.e., sense of belonging and future success), lead to a crisis of representation for women of color and have compounding impacts on the STEM workforce. Research suggests there are positive impacts of in-person STEM learning after-school and out-of-school time programs on girls’ sense of belonging. The increasing need for online learning initiated by the COVID-19 pandemic means it is vital to investigate girls’ STEM identity development within an online community. Thus, the project will refine and test approaches in online learning communities to make a valuable impact on the STEM identity development of girls of color by 1) training educators and role models on exemplary approaches for STEM identity development; 2) implementing a collaborative, girl-focused Brite Online Learning Community that brings together 400 girls ages 13-16 from a minimum of 10 sites across the United States; and 3) researching the impact of the three core approaches -- community building, authentic and competence-demonstrating hands-on activities, and interactive learning with women role models -- on participating girls’ STEM identities in online settings.
The mixed methods study is guided by guided by Carlone & Johnson’s model of STEM identity involving four constructs: competence, performance, recognition, and sense of belonging. Data collection sources for the quantitative portion of the project include pre- and post-surveys, while qualitative data sources will be collected from six case study sites and will include observations, focus group interviews with girls, artifacts created by girls and educators, educator interviews, and open-ended survey responses. This approach will enable the research team to determine how and the extent to which the Brite Online Learning Community influences STEM identity constructs, interpreting which practices lead to meaningful outcomes that can be linked to the development of STEM identity for participating girls in an online environment. The products of this work will include research-based, tested Brite Practices and a toolkit for fostering girls’ interest, identification, and long-term participation in STEM. The resulting products will increase the reach of informal STEM education programming to girls of color across the nation as online spaces can reach more girls, potentially increasing the representation of women of color in the STEM workforce.
Few people realize that the largest part of our planet’s biosphere remains virtually unexplored and unknown. This enormous habitat, accounting for an area of 116 million square miles or the equivalent size of roughly 30 times the area of the United States, is the abyssal zone of the deep ocean. The abyssal sea floor, at about 6000 ft., contains more than four times as much habitat for animal life as all of the dry mountains, forests, deserts, plains and jungles combined. Microscopic larvae in the deep ocean, are essential for the renewal and replenishment of life and they repopulate areas damaged by human activities such as mining and trawling, and they make marine protected areas both feasible and important. The National Science Foundation has funded intensive studies of oceanography related to larval recruitment for decades. However, findings from this large NSF investment of personnel, technology and funding have never been widely presented to the public. This project proposes to remedy this by developing a 40 minute giant screen film to be shown in science centers across the country, supported by virtual reality and augmented reality learning tools. The film will cover select deep ocean science expeditions using the deep-sea vehicles Alvin and ROV Jason. Content will include elements of the research process, activities related to the design and operation of deep-sea vehicles as well as interviews with scientists and technologists. The companion activities, Deep-Ocean Pilot (a VR-360° viewing station) and Plankton Quest (an AR biology treasure hunt) will extend the audience experience of the deep ocean out of the giant screen theater and into the surrounding museum environment. The website and social media will extend awareness and resources into homes. The project will be appropriate for a broad general audience, with particular appeal for the target audience of women and girls (ages 7-20). The larval biologist team is led by the PI at the University of Oregon, in collaboration with scientists from North Carolina State University, Western Washington University and the University of Rhode Island. Several young women scientists will be featured in the film providing role models. The production company, Stephen Low Productions, Inc. will use the latest technology on the Alvin and other cinematic tools to capture the visual images in the abyss. Collaborating museums will participate in the development and implementation of the Virtual and Augmented Reality learning tools as well as showing the film in their theaters.
Broader impact project goals include 1) Advancing public awareness of the abyssal ocean, the role of microscopic larvae, and what scientists are learning from expeditions that use deep submergence technologies; 2) Introducing public audiences and young women specifically to the wide range of STEM-related occupations encompassed in the field of ocean exploration and research; and 3) Advancing STEM learning research and practice in the area of immersive media in conveying STEM concepts and enhancing audience identification with STEM. Oregon State University’s STEM Research Center will build new knowledge by conducting formative and summative evaluation of the film and its associated support products (e.g., Virtual and augmented reality activities, website resources), addressing the following evaluation questions: 1)What do audiences take away from their experience in terms of fascination/interest, awareness and understanding related to ocean science exploration? 2) To what degree does the film alone or in combination with supplemental experiences trigger career awareness in girls and young women, and youth of racial/ethnic backgrounds? 3) To what degree do immersive experiences (a sense of “being there”) contribute to learning from the film? 4) How enduring are outcomes with audiences past the onsite immediate experience? Formative evaluation will be designed as ongoing improvement informed by empirical evidence in which evaluators work with team members to answer decision-relevant questions in a timely and project-focused way. The summative evaluation will be structured as an effectiveness study using mixed methods and ascertaining whether key programmatic outcomes have been reached and the degree to which particular program elements will have contributed to the results.
DATE:
-
TEAM MEMBERS:
Craig YoungAlexander LowStephen LowGeorge von DassowTrish Mace
Data is increasingly important in all aspects of people’s lives, from the day-to-day, to careers and to civic engagement. Preparing youth to use data to answer questions and solve problems empowers them to participate in society as informed citizens and opens doors to 21st century career opportunities. Ensuring equitable representation in data literacy and data science careers is critical. For many girls underrepresented in STEM, developing a "data science identity" requires personally meaningful experiences working with data. This project aims to promote middle school-aged girls’ interest and aspirations in data science through an identity-aligned, social game-based learning approach. The goals are to create a more diverse and inclusive generation of data scientists who see data as a resource and who are equipped with the skills and dispositions necessary to work with data in order to solve practical problems. The research team will run 10 social clubs and 10 data science clubs mentored by women in data science recruited through the University of Miami’s Institute for Data Science and Computing. Participants will be 250 middle school-aged girls recruited in Miami, FL, and Yolo County, CA, through local and national girls’ organizations. Youth will participate in a data science club and will learn key data science concepts and skills, including data structures, storage, exploration, analysis, and visualization. These concepts will be learned from working with their own data collected in personally meaningful ways in addition to working with data collected by others in the same social game eco-system. The project will also develop facilitator materials to allow adult volunteers to create game-based informal data science learning experiences for youth in their areas. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments and is co-funded by the Innovative Technology Experiences for Students and Teachers (ITEST), which seeks to engage underrepresented students in technology-rich learning environments, including skills in data literacy, and increase students’ knowledge and interest in information and communication technology (ICT) careers.
Researchers will focus on two primary research questions: 1) Across gameplay and club experiences, in what ways do participants engage with data to pursue personal or social goals? 2) How do gameplay and club experiences shape girls’ perceptions of data, data science, and their fit with data and data science? The project will use design-based research methods to iteratively design the game and social club experiences. To ensure that uses of data feel personally and socially meaningful to young girls, the virtual world’s goals, narratives, and activities will be co-designed with girls from groups underrepresented in data science. The project will research engagement with game data in two informal, game-based learning scenarios: organic, self-directed, social play club, and structured, adult-facilitated data science clubs. The research will use a combination of quantitative and qualitative methods including surveys, focus groups, interviews, and gameplay and club observations. Project evaluation will determine how gameplay and club experiences impact participants' attitudes toward and interest in data-rich futures. The project holds the potential for broadening participation and promoting interest in data science by blending game-based learning with the rich social and adult mentoring through club participation. The results will be disseminated through conference presentations, scholarly publications, and social media. The game and facilitator materials will be designed for dissemination and made freely available to the public.
DATE:
-
TEAM MEMBERS:
Lisa HardyGary GoldbergerJennifer Kahn
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
The Field Museum of Natural History will present “Changing Face of Science,” an exhibition series targeting pre-teens and teenagers and featuring Field Museum scientists and science educators who are women or people of color. Over three years, the museum will mount six exhibitions that highlight the experiences and work of museum scientists from diverse backgrounds in a range of disciplines. Programming will include on-site field trips and virtual events during which students and educators will interact with featured researchers. By presenting the stories of individuals from groups traditionally underrepresented in scientific fields, the museum will provide role models who will show that science is accessible and inspire a diverse group of future scientists.
This Innovations in Development project aims to foster the development of STEM identity among a diverse group of middle school students and, in turn, motivate them to pursue in STEM interests and careers. Vegas STEM Lab, led by a team of investigators from the University of Nevada, Las Vegas, will employ a mix of online and on-site activities to introduce students to engineering methods in the context of the entertainment and hospitality (E&H) industry that is the lifeblood of Las Vegas. Investigators will collaborate with local resorts, multimedia designers, and arts institutions to offer field experiences for students to interview, interact with, and learn from local experts. The Lab will help youth overcome prevailing beliefs of STEM as boring and difficult, boost their confidence as STEM-capable individuals, and expose them to the exciting STEM careers available in their hometown. UNLV engineering undergrads will serve as near-peer mentors to the middle school students, guiding them through Lab activities and acting as role models. Investigators will measure student learning and engagement over the course of the Vegas STEM Lab experience with the aim of understanding how the Lab model—with its rich set of activities and interpersonal interactions set in the local E&H industry—can cultivate STEM identity development and encourage students to pursue STEM pathways. Despite the project’s hyperlocal focus on the Las Vegas community, if successful, other cities and towns may learn from and adapt the Lab model for use in their youth development programs.
Vegas STEM Lab will provide online materials for students’ STEM learning during the academic year followed by on-site visits and hands-on project development during a three-week summer experience. The Lab will run for three years with cohorts of 40 students each (N=120) with the aim of iteratively improving its activities and outcomes from year to year. The local school district will help recruit middle school students who have demonstrated low interest in STEM to participate in the Lab, ensuring that participants reflect the demographic makeup of the Las Vegas community in terms of race and ethnicity, socio-economic status, and gender. Summer activities will take students behind the scenes of the city’s major E&H venues; investigate the workings of large-scale displays, light shows, and “smart hospitality” systems; and then build their own smaller scale engineering projects. Investigators will employ the Dynamic Systems Model of Role Identity (DSMRI) framework to study how intentionally designed Lab experiences shape students’ understanding of themselves, their future aspirations, and their grasp of the scientific enterprise. Summer activities will be integrated into the online learning platform at the end of each year of Vegas STEM Lab, and in the final year of the project, workshops will train local educators to use the platform in either formal or informal learning settings. Materials and research findings produced through this work will be disseminated to middle school teachers and afterschool care providers, and shared with researchers through academic publications and conferences.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program.
DATE:
-
TEAM MEMBERS:
Emma RegentovaVenkatesan MuthukumarJonathan HilpertSi Jung Kim
Black girls display high interest, confidence and ability in STEM but face multiple barriers including racial, ethnic and gender stereotypes, low exposure to STEM role models, low awareness of diverse STEM fields and financial obstacles to STEM education. It is critical to infuse STEM education with specific and intentional culturally responsive and anti-racist strategies to attract and retain Black girls in STEM. Through this combination of media, role modeling and outreach, Black SciGirls will help increase access to STEM education for Black girls, preparing them for future workforce participation. This project will study the impacts on elementary/middle school Black girls’ exposure to early career Black female STEM professionals as role models. Deliverables include 1) professional development for STEM educators and Black STEM professional women to prepare them to lead STEM programs for girls 2) a PBS series of role model videos of early-career Black STEM professionals and 3) a research study that examines how/if in person and media-based STEM role models increase Black girls’ interest and confidence in STEM, motivation to pursue future STEM studies, and STEM identity. While women make up 47% of the U.S. workforce, they are underrepresented in STEM and only 1.6% are Black women.
The research study will examine how educators’ use of role models addresses a critical barrier for Black girls, seeing women in STEM who look like them. The research study questions are: How and in what ways do Black STEM women role models influence Black girls’ interest in STEM? How and to what extent do role models report changes in their confidence and ability to engage girls in STEM as a result of training in best practices in role modelling? and, How and to what extent are parents engaged in supporting girls’ involvement in STEM, as a result of the participation of role models? The research team will visit participating local SciGirls programs to collect qualitative data, including observations of program activities, interviews, and focus groups. To ensure reliable outcomes and utilize robust theoretical underpinnings, the research will combine pre/post survey data and an in-depth cross-case studies employing qualitative and quantitative data collection. This mixed-methods approach will enable gathering data that comprehensively offers insight into Black girls’ STEM experiences and those of the Black STEM professional women role models and parents who support them. Qualitative data that centers girls’, role models’, and parents’ perspectives will contribute to this identity-centered study. A culturally responsive evaluation will determine the extent to which the project builds educators’ ability to integrate equitable and anti-racist practices to build Black girls’ interest and confidence in STEM studies.
This Research in Service to Practice project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to (a) advance new approaches to and evidence-based understanding of the design and development of STEM learning in informal environments; (b) provide multiple pathways for broadening access to and engagement in STEM learning experiences; (c) advance innovative research on and assessment of STEM learning in informal environments; and (d) engage the public of all ages in learning STEM in informal environments.
DATE:
-
TEAM MEMBERS:
Rita KarlAngel Miles NashRonda BullockAdrienne StephensonLataisia Jones
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
Research shows that Black girls and women, regardless of their academic achievements and STEM interests, often encounter academic under-preparation, social isolation, exclusion, and race-gender discrimination that negatively impacts their ongoing engagement and retention in STEM. This project will provide innovative, culturally relevant learning environments to middle and high school Black girls to counter these negative trends. Using hands-on coding and robotics activities, project participants will develop positive attitudes toward science, technology, engineering, and mathematics (STEM). The project emphasizes peer-mentoring by providing opportunities for Black female high school (assistant coaches) and Black college students (coaches) to serve as counselors and mentors to participants. Additionally, engineers, scientists, and executives from STEM industries will serve as mentors and share their experiences to broaden participants’ STEM career aspirations. The project is a three-year collaborative effort between the University of California Davis C-STEM Center, the Umoja Community Education Foundation, and the 66 affiliated California community colleges, industry partners, and school districts in California. Over three years, nearly 2,000 females will participate in the project.
Learning environments for Black girls and women led by other Black girls and women are referred to as “counterspaces” where they are free to engage in STEM in ways that value their identities while promoting STEM engagement, interests, and career aspirations. The project’s curriculum will follow a research-based, culturally relevant multi-tiered mentoring approach. The curriculum is designed to develop participants’ STEM content knowledge, critical thinking, and logical reasoning capabilities through meaningful connections to real-life applications using hands-on coding and robotics. A mixed-method longitudinal study will examine the impact on participants’ STEM outcomes, emphasizing contributing new knowledge on the viability of multi-tiered, culturally relevant mentoring for increasing equity in informal STEM learning (ISL). The program's effectiveness will be evaluated using longitudinal assessments of mathematics standards, computer science and robotics conceptual knowledge, logical and critical thinking skills, STEM school achievements, interests and attitudes toward STEM subjects, advanced STEM course-taking, involvement in other ISL opportunities, and leadership in STEM in one’s school/university and community. The project will test a locally based informal learning model with projects hosted by other K-12 and college partners.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to (a) advance new approaches to and evidence-based understanding of the design and development of STEM learning in informal environments; (b) provide multiple pathways for broadening access to and engagement in STEM learning experiences; (c) advance innovative research on and assessment of STEM learning in informal environments; and (d) engage the public of all ages in learning STEM in informal environments.
Black Girls Create (BGC) is a program that uses maker pedagogy, social history (i.e., Black women’s contributions in STEM), culturally responsive pedagogy, and mentoring to engage Black girls in STEM. For the project, culturally responsive making is operationally defined as the use of cultural knowledge and maker technologies to create, design, and produce artifacts that are related to a particular concept, theme, or person. The project will conduct a feasibility research study that investigates how engagement in BGC affects participants' STEM interest, STEM confidence, and racial and gender identities. In the project, participants will discover how Black women have made an impact in STEM fields and will learn how to digitally create cultural artifacts/wearable art related to their discoveries. Approximately 120 middle school girls in grades 6-8 (8 groups of 15 participants each) from underserved neighborhoods will be recruited for participation in this two-year study. Each group will meet for 3 hours per week over an 8-week period. Instructional materials from this research will be made freely available online so that they can be adapted and used at other formal and informal educational institutions that seek to garner interest and access to STEM learning for Black girls and women.
A pre- and post-test, quasi-experimental design will be used to research the program’s influence on participants’ self-efficacy, racial identity, and gender identity. Lagged regression models that control for students’ age, race/ethnicity, and pre-survey scores will be used to examine growth in each of the four outcomes. Once all of the participants in the groups have completed BGC, data from all groups will be combined to increase power and thereby detect statistically meaningful differences in pre- and post-survey scores. In addition, variables representing attendance and program engagement will be entered into the model to examine whether students who are highly engaged in BGC programming exhibit more growth in the four outcomes. Qualitative data will be derived from students' journal entries as well as focus group interviews. Weekly journals will be used to gather data about the context and experiences of participants as they unfold throughout the program. Semi-structured focus groups will center around the usefulness of skills and knowledge gained from the program activities, significant experiences with peers and program staff, the opportunity to learn about Black women STEM progenitors, and learning how to make digitally fabricated artifacts. NVIVO, a computer software program, will be used as a tool to support the analysis of the rich, text-based information resulting from the journals and focus group narratives.
This feasibilty research project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to (a) advance new approaches to and evidence-based understanding of the design and development of STEM learning in informal environments; (b) provide multiple pathways for broadening access to and engagement in STEM learning experiences; (c) advance innovative research on and assessment of STEM learning in informal environments; and (d) engage the public of all ages in learning STEM in informal environments.
This award takes an innovative approach to an ongoing, pervasive, and persistent societal issue: women are still drastically underrepresented in computing careers. This project targets middle school-aged girls because it is a time when many of them lose interest and confidence in pursuing technical education and computing careers. This project will design, develop, and deploy a one-week experience focused on middle school girls that targets this issue with a novel combination of teaching techniques and technology. The project will use wearable computing devices to support girls' social interactions as they learn computing and solve technical challenges together. The goals of the project are to raise interest, perceived competence, and involvement in the computational ability of girls. Additionally, the project aims to increase a sense of computational community for girls that makes pursuing computational skills more relevant to their identities and lives, and that helps continued participation in computing. The project will deploy a one-week experience four times per year with a socioeconomically diverse range of campers. The project will also develop a 'program in a box' kit that can be broadly used by others wishing to deliver a similar experience for girls.
The planned research will determine if a one-week experience that uses social wearable construction in the context of live-action role play can use the mediating process of computational community formation to positively impact middle school girls' engagement with and interest in computation. Computational community is defined as girls engaging together in the process of learning computation, trading resources and knowledge, and supporting growth. Research participants will include 100 6th to 9th-grade girls. At least 75% of the participants will be either low income, first-generation college-bound, or underrepresented in higher education. Students will be recruited through the longstanding partnerships with title one schools in the Salinas Valley, the Educational Partnership Center, and in the Pajaro Valley Unified School district, where 82% of the students are Hispanic/Latinx, 42% are English Learners, and 73% are eligible for free or reduced lunch. The research questions are: 1) Does the proposed experience increase girls' self-reported competence, self-efficacy, and interest in computational skills and careers? and 2) Will the proposed experience lead to activity-based evidence of learning and integration of computational skills at the group social level? The project will use a mixed-methods, design-based research approach which is an iterative design process to rapidly collect and analyze data, and regularly discuss the implications for practice with the design team. Data will be collected using observations, interviews, focus groups, surveys, and staff logs. Quantitative data will be analyzed using frequencies, means, and measures of dispersion will be applied to survey data from both time points. Pearson correlation coefficients will be used to describe the bivariate relationship between continuous factors. ANOVAs will assess whether there are significant differences in continuous measures across groups. Qualitative data will be analyzed using a constant comparison method.
This Innovations in Development award is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.