This is the final evaluation report from RMC Research Corp. for the PES@LTERs project. Appendix includes instruments.
RMC Research designed evaluation activities to provide formative and summative feedback to Harvard Forest and the Hubbard Brook Research Foundation (Hubbard Brook) on their plan to embed public engagement with science (PES) into the cultures and practices of Long-Term Ecological Research Sites (LTERs) in the northeastern US. The purpose of this project was to build PES mechanisms into long-term ecosystem studies that create on-going, open exchanges between scientists and
Despite the fact that most science learning takes place outside of school, little is known about how engagement in informal science learning (ISL) experiences affects learners’ knowledge, skill development, interest, or identities over long periods of time. Although substantial ISL research has documented short-term outcomes such as the learning that takes place during a science center visit, research suggests that the genuine benefits of informal experiences are long-term transformations in learners as they pursue a “cascade” of experiences subsequent to the initial educational event
Persistent racial injustices and inequities in the United States and in STEM fields underscore the need for creative, research-based approaches to address these concerns. In particular, creative approaches are needed for studying and addressing racial injustices and inequities in STEM education, where racial equity and STEM learning are both given careful and thoughtful consideration. This project focuses on supporting emerging scholars who have new ideas and approaches for approaching racial equity in their scholarship and work. This workshop, implemented as a series of sessions over the course of a year, will support early career scholars in STEM education and the learning sciences in preparing proposals to submit to the National Science Foundation. The workshop is designed to serve scholars who are within five years of obtaining their PhD and who have never before been principal investigator or co-principal investigator of a federally-funded grant. Participants will include early career scholars who focus their work on racial equity. Too often, such scholars have indicated that they have received little to no training on writing grant proposals.
Ten participants will be supported by the project through a year-long series of workshops that include different aspects of the grant writing process including reading through a solicitation, writing a narrative, and creating a budget. In addition to these workshop sessions, the project approach also considers the importance of a professional network and of mentoring, informed by a Communities of Practice theoretical framework and existing research on mentoring practices. As such, each early career scholar will be paired with a senior mentor in the field whose work is aligned with the mentee's. The outcomes of the workshop for early career scholars will include a complete or nearly complete proposal that is aligned with one of the programs within the NSF's Division of Research on Learning. The workshop will highlight strategies for developing CAREER proposals along with considerations for preparing proposals for other programs. More generally, the workshop will create a model for supporting and mentoring early career scholars in proposing STEM education projects centered in racial equity work and will be able to identify areas of need for successful grant proposal writing. All workshop materials will be made freely available to the general public.
The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This project is also funded through the Advancing Informal STEM Learning (AISL), CS for All: Research and RPPs, and Innovative Technology Experiences for Students and Teachers (ITEST) programs.
HBCUs are critical to producing a diverse and inclusive workforce as they graduate a disproportionate number of African American future STEM workers and STEM leaders. Although the National Science Foundation is fully committed to diversity and inclusion, there has been little research to determine why Historically Black Colleges and Universities are not fully participating in the NSF STEM educational research opportunities. The project will investigate the challenges, needs and support for Historically Black Colleges and Universities (HBCUs) to succeed in applying for educational research support from the National Science Foundation (NSF). Participants will be recruited from 96 HBCUs that are eligible to apply for such funding and will include the wide range of college and university administration and faculty that are involved in the preparation of research projects and related applications for research funding. The investigation will focus primarily on the Division of Research on Learning in Informal and Formal Settings (DRL) within NSF. The investigation will: 1) determine the submission rate and funding success rate of HBCUs within the DRL funding mechanisms; 2) determine why a greater proportion of HBCUs are not successful in their applications of research or do not apply; and 3) determine what factors, such as institutional support, research expertise, and professional development, could lead to a larger number of research proposals from HBCUs and greater success in obtaining funding. The project has the potential to have significant influence on the national educational and research agenda by providing empirical findings on the best approach to support and encourage HBCU participation in DRL educational research funding programs.
This exploratory research project will investigate what changes and/or supports would contribute to significantly increasing the number of applications and successful grant awards for STEM educational research project proposed by HBCUs. The project has the following research questions: (1) What factors discourage participation of HBCUs in the DRL funding mechanisms and what are the best practices to encourage participation? (2) What approaches have been successful for HBCUs to obtain DRL funding? (3) What dynamic capabilities are necessary for HBCU researchers to successfully submit STEM proposals to NSF? (4) What changes would be helpful to reduce or eliminate any barriers for HBCU applications for DRL educational research funding and what supports, such as professional development, would contribute to greater success in obtaining funding? Participants will be recruited from the 96 eligible HBCUs and will include both individuals from within the administration (e.g., Office Sponsored Programs, Deans, VP, etc.) as well as from within the faculty. The research will collect variety of quantitative and qualitative data designed to support a comprehensive analysis of factors addressing the research questions. The project will develop research findings and recommendations that are relevant to faculty, administrators, and policymakers for improving HBCU participation in research funding opportunities. Results of project research will be widely disseminated to HBCUs and other Minority Serving Institutions (MSIs) through a project website, peer reviewed journals, newsletters, and conference presentations.
This project is funded by the Innovative Technology Experiences for Students and Teachers (ITEST), the Advancing Informal STEM Learning (AISL), and the Discovery Research PreK-12 (DRK-12) programs. These programs which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' and general public knowledge and interest in science, technology, engineering, and mathematics (STEM).
DATE:
-
TEAM MEMBERS:
Cynthia TrawickJohn HaynesTriscia HendricksonTerry Mills
Research suggests that when both science, technology, engineering, and mathematics (STEM) education and social-emotional development (SED) are supported in afterschool, summer, and other informal settings, young people can better develop skills for the future such as leadership, decision-making, and relationship-building so they could have successful careers/participation in STEM. However, researchers and practitioners working in the out-of-school time (OST) sector often do so without connections across these fields. The appeal for more integration of STEM and SED in OST program delivery and data collection has remained abstract and aspirational. This Literature Review and Synthesis project is the next step needed to move the OST field toward the intentional, explicit, and evidence-based integration of STEM and SED in research and practice. The project will create shared understanding necessary to improve program content, staff training, and evaluation. This synthesis will support future research on unified STEM+SED that can lead to more effective, equitable, and developmentally appropriate programming. Improved programming will contribute to talent development, address STEM workforce needs, and promote socioeconomic mobility to benefit children, youth, educators, and society. This project is funded by the Advancing Informal STEM Learning (AISL) program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
This project will systematically examine what domains and skills at the interface of STEM+SED are most researched among K-12 youth in informal STEM learning environments, compared to formal STEM educational environments. The team will further explore how gender, race, and other intersectional forms of equity can be added to the STEM+SED equation. The project team will search and appraise empirical and gray literature (2001-2020) to identify the most commonly researched domains and skills at the interface of STEM+SED in informal environments serving K-12 youth. The review and synthesis process will include four steps: search, appraisal, synthesis, and analysis. The search will begin with STEM+SED skills in four foundational domains (agency, belonging, engagement, and reflection) identified previously with experts from the fields of STEM and SED. The search will include all existing, eligible references from formal K-12 settings to contrast commonly studied domains and skills (e.g., perseverance, self-regulation, teamwork, complex problem-solving, self-awareness) in formal versus informal learning environments. The study approach will then compare these domains and skills by the demographics variables noted above. Following the creation of a strong catalog of evidence, information will be synthesized using three “pillars” for building coherence in STEM+SED integration: phenomenon (the knowing), implementation (the doing) and assessment (the result). These pillars will be used to organize and critically analyze the literature. Building conceptual coherence through a systematic review and synthesis of literature from the fields of STEM and SED will lead to greater understanding of STEM+SED in OST practice, highlight the most important content and skills to learn in informal environments, and identify when and how youth should learn specific content and skills at the interface of STEM+SED. Applying coherence to the integration of STEM+SED ensures that the principles and practices are layered carefully, in ways that avoid superficial checklists or duplication of effort and build meaningfully upon young people’s knowledge and skills. The long-term goal is to broker connections and alignment of STEM+SED across schools and OST programs. Recommendations and a roadmap to guide equitable, effective STEM+SED research, practice, and policy will result from this research.
Informal STEM learning environments, programs, and policies can be designed to support and promote neurodiversity through inclusive practices. This project will explore the benefits of informal STEM learning for K-12 neurodiverse learners through a systematic review and meta-analysis of extant literature and research grounded in the theory of social model of ability. This framework is an asset-based approach and aims to promote social, cognitive, and physical inclusion, leading to positive outcomes. Using various quantitative and qualitative methodologies, this project endeavors to collect and synthesize the evidence for supporting and enhancing accessibility and inclusiveness in informal STEM learning for K-12 neurodiverse learners. It will explore key features of informal STEM learning and effective, evidence-based strategies to effectively engage children and youth with neurological conditions such as autism spectrum disorder, attention deficit hyperactivity disorder (ADHD), dyslexia, and dyspraxia, in informal STEM learning environments. The findings of this complex synthesis will provide a timely contribution to deeper understanding of supports for neurodiversity while also highlighting areas that inform further research, shifts in practice, and policy.
The systematic review will occur over a two-year period. It will focus on identifying program elements that promote inclusion of children and youth with neurodevelopmental disabilities in informal STEM learning contexts. Specifically, the review will explore two overarching research questions and several sub-research questions:
RQ1. What program elements (teaching and learning variables) in informal STEM learning settings facilitate inclusion of K-12 neurodiverse STEM learners? Sub-RQ1a: What are the overlapping and discrete characteristics of the program elements that facilitate social, cognitive, and physical inclusion?
Sub-RQ1b: In what ways do the program elements that facilitate inclusion vary by informal STEM learning setting?
RQ2: What program elements (teaching and learning variables) in informal STEM learning settings are correlated with benefits for K-12 neurodiverse STEM learners? Sub-RQ2a: What are the overlapping and discrete characteristics of the program elements that correlate with increased STEM identity, self- efficacy, interest in STEM, or STEM learning?
Sub-RQ2b: In what ways do the program elements that correlate with positive results for students vary by informal STEM learning setting? The research synthesis will consider several different types of studies, including research and evaluation; experimental and quasi-experimental designs; quantitative, qualitative, and mixed methods; and implementation studies.
The research team will (a) review all analyses and organize findings to illustrate patterns, factors, and relationships, (b) identify key distinctions and nuances derived from the contexts represented in the literature, and (c) revisit and confirm the strength of evidence for making overall assertions of what works, why, and with whom. The findings will be disseminated in practice briefs, journal articles, the AISL resource center, as well as presentations and materials for researchers, practitioners, and informal STEM leaders. Ultimately, this work will result in a comprehensive synthesis of effective informal STEM learning practices for neurodiverse K-12 learners and identify opportunities for further research and development.
This literature review and meta-analysis project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This presentation "Revealing Impacts: How does one long-term, statewide field trip program influence a community's learning ecosystem?" was shared as showcase at the 2021 Connected Learning Summit (https://app.clowdr.org/conference/clsummit2021/) held virtually from July 7-30, 2021.
In the showcase session, we shared details of an NSF funded collaborative project between the Gulf of Maine Research Institute, Oregon State University and TERC to understand how a longstanding, statewide field trip program, LabVenture, influences a community’s learning ecosystem. The project uses the construct
The Council for Opportunity in Education, in collaboration with TERC, seeks to advance the understanding of social and cultural factors that increase retention of women of color in computing; and implement and evaluate a mentoring and networking intervention for undergraduate women of color based on the project's research findings. Computing is unique because it ranks as one of the STEM fields that are least populated by women of color, and because while representation of women of color is increasing in nearly every other STEM field, it is currently decreasing in computing - even as national job prospects in technology fields increase. The project staff will conduct an extensive study of programs that have successfully served women of color in the computing fields and will conduct formal interviews with 15 professional women of color who have thrived in computing to learn about their educational strategies. Based on those findings, the project staff will develop and assess a small-scale intervention that will be modeled on the practices of mentoring and networking which have been established as effective among women of color who are students of STEM disciplines. By partnering with Broadening Participation in Computing Alliances and local and national organizations dedicated to diversifying computing, project staff will identify both women of color undergraduates to participate in the intervention and professionals who can serve as mentors to the undergraduates in the intervention phase of the project. Assisting the researchers will be a distinguished Advisory Board that provides expertise in broadening the representation of women of color in STEM education. The external evaluator will provide formative and summative assessments of the project's case study data and narratives data using methods of study analysis and narrative inquiry and will lead the formative and summative evaluation of the intervention using a mixed methods approach. The intervention evaluation will focus on three variables: 1) students' attitudes toward computer science, 2) their persistence in computer science and 3) their participant attitudes toward, and experiences in, the intervention.
This project extends the PIs' previous NSF-funded work on factors that impact the success of women of color in STEM. The project will contribute an improved understanding of the complex challenges that women of color encounter in computing. It will also illuminate individual and programmatic strategies that enable them to participate more fully and in greater numbers. The ultimate broader impact of the project should be a proven, scalable model for reversing the downward trend in the rates at which women of color earn bachelor's degrees in computer science.
This research extends the investigator's prior NSF supported work to develop theoretical and empirical understanding of the double bind faced by women of color in STEM fields. That is, their race and gender present dual dilemmas as they move through STEM educational and career paths. The proposed study will identify gaps in our understanding, and identify some of the methodological problems associated with answering outstanding questions about the double bind. The major research question is: What strategies work to enable women of color to achieve higher levels of advancement in STEM academia and professions? The goal is to bring a clearer understanding of the issues which confront women of color as they pursue study of science and engineering, and what factors influence whether they leave or remain in STEM.
The work will employ a highly structured narrative analysis process to identify and quantify factors that have been successful in broadening the participation of minority women in STEM. The research design involves two separate tracks of work: 1) to conduct narrative analysis of primary documents associated with women of color in science; and 2) to conduct site visits and interviews to understand features of programs associated with successful support of women of color in undergraduate and graduate education. The first part is designed to inform the second, with the narrative analysis helping to identify features to look for in site visits and to use in development of interview protocols.
This research will focus on individual and programmatic factors that sustain women of color as they confront barriers to their career goals. It examines institutional strategies and support structures that help women of color ultimately to succeed, and social and pedagogic elements that influence their educational experiences. Although women of color have made some progress over the last three decades towards more equitable participation in STEM fields, the major efforts made to address this issue have not produced the desired outcomes; minority women continue to be underrepresented relative to white women and non-minority men. The factors that account for continued lower participation rates are not yet fully understood.
Beyond the Double Bind is designed to transform the intellectual basis for building future programs that will better enable women of color to be successful in STEM. While focused on women of color, the results will ultimately inform strategies and programs to expand the presence of all women and minorities in STEM.
This workshop is funded through the "Dear Colleague Letter: Principles for the Design of Digital Science, Technology, Engineering, and Mathematics (STEM) Learning Environments (NSF 18-017)." In today's educational climate, organizations are creating physical learning spaces for hands-on STEM activities, often called makerspaces, co-working spaces, innovation labs, or fablabs. These spaces have evolved to be interdisciplinary centers that personalize learning for individual, diverse learners in collaborative settings. When designed well, these physical spaces create communities that contextualize learning around participants' goals and thus address STEM learning in a dynamic and integrated way. Participation in these learning environments encourages the cultivation of STEM identities for young people and can positively direct their career trajectories into STEM fields. This workshop will bring together a community of collaborators from multiple stakeholder groups including academia, public libraries, museums, community based organizations, non-profits, media makers and distribution channels, and educators within and beyond K-12 schools. Led by the University of Arizona, and held at Biosphere 2, an international research facility, participants will engage in activities that invite experimentation with distributed learning technologies to examine ways to adapt learning to the changing technological landscape and create robust, dynamic online learning environments. The workshop will culminate in a synthesis of design principles, assessment approaches, and tools that will be shared widely. Partnerships arising from the workshop will pave the way for sustained efforts in this area that span research and practice communities. Outcomes will address research and development of the next generation of digitally distributed learning environments.
The three day workshop convening will provide a unique forum to (1) exchange innovative ideas and share challenges and opportunities, (2) connect practical and research-based expertise and (3) form cross-institutional and cross-community partnerships that envision, propose, and implement opportunities for collecting and analyzing data to systematically inform the collective understanding. Participation-based activities will include design-based experiences, participatory activities, demonstrations of works in progress, prototyping, creative pitching, practitioner lightning talks, small group breakouts, hands-on design activities, and an 'unconference' style synthesis of bold ideas. Participants will be invited to experiment with distributed learning technologies. Five focus areas for the workshop include (1) inclusivity of learning spaces that invite multiple perspectives and full participation, (2) documenting learning in ways that are linked to outcomes and impacts for all learners, (3) implementing the use of new technologies in diverse settings, such as the workforce, (4) interpersonal interactions and peer-to-peer learning that may encourage a STEM career-path, and, (5) methods for collecting and analyzing data at the intersection of people, the learning environment, and new technologies at multiple levels. Outcomes of the workshop will serve to advance knowledge regarding critical gaps and opportunities and identify and characterize models of collaboration, networking, and innovation that operate within and across studio-based STEM learning environments.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.