Informal STEM education needs high quality program evaluation. Research is particularly needed on the relationship between STEM outcomes and positive youth development (PYD) / socio-emotional learning (SEL). This pilot and feasibility project involves a collaboration between experts in STEM education, out-of-school time programs (OST), PYD, SEL, evaluation, and program development. STEM Scouts helps youth grow in character and develop skills using experiential activities and interaction with STEM professionals. This project expands the implementation of the Systems Evaluation Protocol (SEP), an evidence-based approach to evaluation planning.
The goals are to: 1) develop a theory of change for STEM Scouts highlighting the relationship between STEM outcomes and PYD/SEL outcomes; 2) pilot enhancements to the SEP (System Mapping, Ecosystem Modeling, and Model Validation); and, 3) determine the feasibility of conducting a national STEM Scouts study. STEM Scouts leadership and project researchers will work through the SEP to generate a stakeholder map, logic model, and pathway model (PM). Five STEM Scouts Labs across the country will participate in focus groups where they will complete SM to identify the system in which the Lab exists (e.g., stakeholders and decision-makers), review and revise their system maps, identify key outcomes and connections in the PM, and discuss how the PM reflects their STEM Scouts experiences. It is hypothesized the enhanced SEP will enable the working group to better understand factors hindering or enabling program and evaluation feasibility and success. Findings will be disseminated to the evaluation/research community, OST program providers/developers and the public.
This project is funded by the National Science Foundation's Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Gathering evidence for the long-term impact of programs for youth on their involvement in STEM studies and careers continues to be a challenge, especially for program interventions happening in earlier stages of development (e.g., elementary, middle, high-school years. Work that focuses on mechanisms for mitigating these challenges is important. The conference and associated activities aim to build a research action agenda that is rooted in practice to support better understanding of the long-term impacts of informal STEM programs for girls provided by cultural institutions, along with methods and approaches for measuring them. The project team will use a collaborative co-design approach to establish a STEM for Girls Research Alliance. The Alliance will include three levels of participants, with different levels of commitment: a core planning group (CPG) of 8-10 people, approximately 20-25 participating members (PM) and 50-60 interested stakeholder group representatives (SGR). The project team will utilize face-to-face meetings, digital engagement strategies, and surveys to develop the agenda and solicit multiple rounds of feedback and input. The CPG, consisting of leaders and representatives from state-based STEM for Girls organizations that are part of the National Girls Collaborative Project (NGCP) and members of the New York State STEAM for Girls Collaborative, will be responsible for setting priorities and guiding the action agenda. The PM will include representatives (educators and researchers) from informal STEM programs at cultural institutions that participate in the state-based collaboratives. The PM will be regularly consulted on important aspects of the action agenda that relate to their work. Finally, the SGR will include representatives from several audiences that are being served by or work with the participating members: girls and young women, parents, educators, funders, researchers and employers. The SG will be engaged via focus groups virtually or at national meetings to which these individuals attend.
To support broader involvement of professionals working in this sector, a comprehensive digital engagement plan using web and social media networks will be developed. The plan will utilize a consistent hashtag, #STEMeffect, allowing participants to follow the conversation across social media platforms. Social media platforms to be utilized will include Twitter, Instagram, Facebook, LinkedIn, Google+, Snapchat and others. More than 60,000 people will be engaged via the networks of the NY STEAM Collaborative, NGCP, WSKG Public Media and project partners. A robust research action plan will position researchers to further explore the role of informal STEM education experiences in shaping the career choices of girls and identify where there are breaks in the hoped-for pathways to STEM college and careers. It also will benefit informal STEM organizations by yielding information that will help them to fine-tune their programs for girls and young women. Ultimately, contributions to the knowledge base will result in broadened participation of girls and women in STEM programs and careers. This work is funded by the Advancing Informal STEM Learning (AISL) program as part of its strategy to enhance learning in informal environments and support innovative research, approaches, and resources for use in a variety of settings.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Lynda KennedyAlicia SantiagoBabette Moeller
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Museum of Science, Boston (MOS) and Boston University (BU) will conduct a Pilot and Feasibility Study project that leverages the current Living Laboratory (LL) model and expand it to engage high school students (teens) in experimental psychology research, science communication and science education activities. In LL, which is now an extensive network of museums and university researchers across the country, scientists and museum staff collaborate to engage children in studies on the museum floor and educate caregivers about the research. Multi-site implementation and evaluation of LL has also documented positive impacts for undergraduate researchers. Many sites are eager to extend these benefits to high school students by engaging them as practitioners within the model and by providing them with opportunities to engage in current research, education and communication, thereby helping to foster stronger youth identities with science and its applications in society. This project expands a ten-year LL partnership between MOS and BU to: 1) pilot a program in which high school students both conduct scientific research and engage the public in learning about science; 2) explore strategies for museums and universities to collaboratively engage, support and mentor high school students in science research, communication and education activities; 3) document curricular, other programmatic, and evaluation materials; and 4) convene professional participants to provide feedback on pilot materials, and assess the viability of implementing similar programs at additional sites. Guided by developmental evaluation, these activities will generate knowledge for the field, and act to increase professional capacity to integrate experiences for teens at multiple LL sites in future projects. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project is a two-day conference, along with pre- and post-conference activities, with the goal of furthering the informal science learning field's review of the research and development that has been conducted on data visualizations that have been used to help the public better understand and become more engaged in science. The project will address an urgent need in informal science education, providing a critical first step towards a synthesis of research and technology development in visualization and, thus, to inform and accelerate work in the field in this significant and rapidly changing domain.
The project will start with a Delphi study by the project evaluator prior to the conference to provide an Emerging Field Assessment on data visualization work to date. Then, a two-day conference at the Exploratorium in San Francisco and related activities will bring together AISL-funded PIs, computer scientists, cognitive scientists, designers, and technology developers to (a) synthesize work to date, (b) bring in relevant research from fields outside of informal learning, and (c) identify remaining knowledge gaps for further research and development. The project team will also develop a website with videos of all presentations, conference documentation, resources, and links to social media communities; and a post-conference publication mapping the state of the field, key findings, and promising technologies.
The initiative also has a goal to broaden participation, as the attendees will include a diverse cadre of professionals in the field who contribute to data visualization work.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Engagement is the cornerstone of learning in informal science education. During free-choice learning in museums and science centers, visitor engagement shapes how learners interact with exhibits, navigate through exhibit spaces, and form attitudes, interests, and understanding of science. Recent advances in multimodal learning analytics are creating novel opportunities for expanding the range and richness of measures of visitor engagement in free-choice settings. In particular, multimodal learning analytics offer significant potential for integrating multiple data sources to devise a composite picture of visitors' cognitive, affective, and behavioral engagement. The project will center on providing a rich empirical account of meaningful visitor engagement with interactive tabletop science exhibits among individual visitors and small groups, as well as uncovering broader tidal patterns in visitor engagement that unfold across exhibit spaces. A key objective of the project is creating models and practitioner-focused learning analytic tools that will inform the best practices of exhibit designers and museum educators. This project is funded by the Advancing Informal STEM Learning (AISL) program. As part of its overall strategy to enhance learning in informal environments, AISL funds research and innovative approaches and resources for use in a variety of settings. The research team will conduct data-rich investigations of visitors' learning experiences with multimodal learning analytics that fuse the rich multichannel data streams produced by fully-instrumented exhibit spaces with the data-driven modeling functionalities afforded by recent advances in machine learning and educational data mining. The research team will conduct a series of visitor studies of naturalistic engagement in solo, dyad, and group interactions as visitors explore interactive tabletop science exhibits. The studies will utilize eye trackers to capture visitors' moment-to-moment attention, facial expression analysis and quantitative field observations to track visitors' emotional states, trace logs generated by exhibit software, as well as motion-tracking sensors and coded video recordings to capture visitors' behavioral interactions. The studies will also use conversation recordings and pre-post assessment measures to capture visitors' science understanding and inquiry processes. With these multimodal data streams as training data, the research team will use probabilistic and neural machine learning techniques to devise learning analytic models of visitor engagement. The project will be conducted by a partnership between North Carolina State University and the North Carolina Museum of Natural Sciences. The research team will 1) design a data-rich multimodal visitor study methodology, 2) create the Visitor Informatics Platform, a suite of open source software tools for multimodal visitor analytics, and 3) launch the Multimodal Visitor Data Warehouse, a curated visitor experience data archive. Together, the multimodal visitor study methodology, the Visitor Informatics Platform, and the Multimodal Visitor Data Warehouse will enable researchers and practitioners in the informal science education community to utilize multimodal learning analytics in their own informal learning environments. It is anticipated that the project will advance the field of informal STEM learning by extending and enriching measures of meaningful visitor engagement, expanding the evidence base for visitor experience design principles, and providing learning analytic tools to support museum educators. By enhancing understanding of the cognitive, affective, and behavioral dynamics underlying visitor experiences in science museums, informal science educators will be well-positioned to design learning experiences that are more effective and engaging. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
James LesterJames MinogueJonathan RoweNorth Carolina Museum of Natural Sciences
resourceresearchProfessional Development, Conferences, and Networks
Leading science educators from 9 South and Southeastern Asian countries and the U.S. met for three days in Kuala Lumpur, Malaysia (October 4-6, 2017) in an effort to rethink and re-envision science education in the 21st Century. The attendees of this U.S. National Science Foundation-funded international conference reaffirmed the G8-Science Academies Joint Statement (2011) that education in science must be targeted not only to future scientists, engineers and other specialists in government and industry, but also to the general public, including school-aged children and adults. The attendees at
The San Diego Natural History Museum contracted RK&A to conduct an evaluation of the SPECTRUM Social Stories Project. The program involved a series of 16 “meetups” for museum staff and young adults with an autism spectrum disorder (ASD) to visit seven Balboa Park museums. The young adults, with support from museum staff, used their observations of those spaces to create pre-visit materials for museum-goers with ASD.
How did we approach this study?
The evaluation explored outcomes for young adult participants, museum staff, and project partners from other Balboa Park museums. We conducted
Supported by the National Science Foundation, the Global Soundscapes! Big Data, Big Screens, Open Ears project employs a variety of informal learning experiences to present the physics of sound and the new science of soundscape ecology. The interdisciplinary science of soundscape ecology analyzes sounds over time in different ecosystems around the world. The major components of the Global Soundscapes project are an educator-led interactive giant-screen theater show, group activities, and websites. All components are designed with both sighted and visually impaired students in mind. Multimedia
This is a report from Workshop IV, part of the Support Systems for Scientists' Communication and Engagement workshop series. Workshop IV was held May 2-3, 2018 at the Monterey Bay Aquarium in Monterey, CA. In this study, researchers sought to explore and understand the mechanisms, motivations, and outcome metrics utilized by individuals and organizations that facilitate scientists’ communication with the public. To capture the full range of engagement methods, facilitators from a wide variety of organizations were contacted and interviewed. After contacting 30 leaders the realm of science
Science in the Learning Gardens (henceforth, SciLG) program was designed to address two well-documented, inter-related educational problems: under-representation in science of students from racial and ethnic minority groups and inadequacies of curriculum and pedagogy to address their cultural and motivational needs. Funded by the National Science Foundation, SciLG is a partnership between Portland Public Schools and Portland State University. The sixth- through eighth-grade SciLG curriculum aligns with Next Generation Science Standards and uses school gardens as the milieu for learning. This
DATE:
TEAM MEMBERS:
Dilafruz WilliamsHeather Anne BruleSybil Schantz KelleyEllen A. Skinner
In this paper, we summarize the results of the two-year, National Science Foundation-funded Head Start on Engineering (HSE) project, designed to study and support engineering-related interest development for preschool children and their families from low-income backgrounds participating in Head Start. Low-income communities face ongoing barriers to accessing STEM learning resources and pursuing STEM-related careers. Quality family interventions in early childhood are a critical approach to addressing these barriers and have been shown to have long-term, positive impacts on families well beyond
This is a report from Workshop II: Overview of Organizational Support for Public Engagement among Scientific Societies, part of the Support Systems for Scientists' Communication and Engagement workshop series. Workshop II was held Feb. 28 - Mar. 1, 2018 at the Howard Hughes Medical Institute in Chevy Chase, MD. The report provides an overview of a project aiming to help reveal how societies view the concept of “public engagement” and to provide an empirical understanding of the availability and prevalence of tools that societies currently use to support engagement efforts.
DATE:
TEAM MEMBERS:
Shupei YuanAnthony DudoJohn BesleyBrooke Smith