The National Science Foundation's (NSF)Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES) program supports models, networks, partnerships and research to ensure the broadening participation in STEM of women, members of racial and ethnic groups that have been historically underrepresented, persons of low socio-economic status, and people with disabilities.
The University of Akron will convene a two-day conference to develop a backbone organization to support the preparation and advancement of underrepresented minorities K-12 through careers in the biosciences, a high growth area for engineering (biomechanics, biometrics and biomaterials). This conference draws on the expertise of a wide range of organizations, professional associations, K-20+, community based organizations, industry and museums. The intent is to strengthen the network among participants and leverage learning on how to engage youth in the biosciences.
The results of this first conference will be a white paper that will be disseminated to several professional societies that outlines a backbone infrastructure for addressing both short-term and longer-term aspects of an NSF INCLUDES alliance centered on bioengineering, biomechanics, biomedical engineering and biomaterials.
DATE:
-
TEAM MEMBERS:
Brian DavisCarin HelferRouzbeh Amini
resourceprojectProfessional Development, Conferences, and Networks
This project will develop, organize and host a Conference to assist in the planning of the NSF INCLUDES (Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science) Alliances and National Network. The focus of this project is the design and implementation of program evaluations necessary for the INCLUDES Alliances and National Network. The conference will be the first to emphasize the multiple scales at which evaluations occur for programs that include components based at single institutions, at interdisciplinary centers/institutes within larger institutions, at alliances/partnerships between several institutions and at entities with a national scope. As the INCLUDES Alliances and National Network will operate across multiple scales, the proposed Conference will provide guidance on the technical aspects of designing and planning multi-scale evaluations.
The target audience for the activities are those individuals involved in current INCLUDES projects, those considering collaborating in such projects and STEM educators considering inclusion of formal evaluation in their projects. The overall goals are to (i) enhance the knowledge of the participants about evaluation methods; (ii) present the experiences of individuals who have successfully developed alliances and carried out evaluation efforts for these; and (iii) provide advice regarding evaluation methods for those planning to participate in future requests for INCLUDES Alliances and/or the National Network. The methods proposed include three components: (a) an initial webinar open to the broad community of STEM educators to introduce key concepts and vocabulary concerning evaluation methods; (b) a pre-Conference Tutorial with 30 participants to provide a one-day overview of modern methods in evaluation and connect these to projects suggested by the participants; and (c) a two-day Conference with 80 participants including as speakers leading experts on program evaluation as well as leaders of programs in STEM education that have operated at multiple scales. Virtual participation will be integrated throughout the three components of the project and a formal evaluation of the project will be carried out using the Systems Evaluation Methodology that has been in use at the host NSF-sponsored Center (NIMBioS) for many years.
DATE:
-
TEAM MEMBERS:
Louis GrossSuzanne LenhartErnest BrothersPamela Bishop
This award supports a conference that will inform the design of "backbone" organizations for the NSF INCLUDES (Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science) program. NORC at the University of Chicago (an independent research institution that delivers reliable data and rigorous analyses to guide programmatic and policy decisions) and TERC (a nonprofit education research and development organization based in Cambridge, MA, dedicated to improving STEM learning for all) collaborate on Envisioning Impact, a two-part in-person and virtual event that will inform the design of INCLUDES Alliance and National Network backbone organizations.
The objectives of the conference are to: (1) facilitate a shared vision of impact for INCLUDES broadening participation projects and program; (2) stimulate discourse on key elements of a shared measurement system for continuous improvement and outcome assessment; and (3) inform decisions on the infrastructure and priority services INCLUDES backbone organizations will provide to assist grantees (and others) in assessing progress towards collective impact. The conference will bring members of three communities together: PIs and evaluators of INCLUDES Design and Development Launch Pilots; investigators and evaluators of other NSF-supported broadening participation alliances, extension projects, and other collective efforts to support inclusion and diversity in STEM; and members of prior and extant NSF-supported knowledge-networking, collaboratory, and resource network initiatives. Members of these communities will collaborate in two separate events: an in-person, 1.5 day conference, and a follow-on virtual Video Hall that will allow a larger number of participants to engage, over a one week period, in facilitated community discourse around short video narratives produced by each project.
DATE:
-
TEAM MEMBERS:
Kevin BrownSarah-Kathryn McDonaldJoni Falk
resourceprojectProfessional Development, Conferences, and Networks
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This conference proposal represents the first phase of a larger three-phase participatory research project that will use communities of interest as a vehicle for solving problems of common concern about designing youth-based STEM programs. It will set the stage for research over the next 10-25 years about the long-term impact of a variety of youth programs on STEM learning and career aspirations. Through a virtual format, the Association of Science-Technology Centers will bring together two representatives from ten long-standing youth programs, experts in the field of out-of-school time youth programming, and researchers to collaboratively develop a program profile template for measuring the impact of youth programming. The program profile template will help identify specific characteristics that will capture the influence of youth programs on their participation in out-of-school STEM activities.
The program profile template will be the main outcome from the conference. It will serve as the foundation for designing long-term impact studies that support the needs of program staff interested in improving youth programming in informal environments. It will also allow program staff and researchers to document and share intellectual capital, compare goals and features across programs, and support network efforts among informal agencies worldwide. The program profile template will be shared online through informalscience.org, the Association of Science-Technology Centers' communities of practice networks, and through other out-of-school-time national organizations.
This project was submitted in response to EHR Core Research (ECR) program announcement NSF 15-509. The ECR program of fundamental research in STEM education provides funding in critical research areas that are essential, broad and enduring. EHR seeks proposals that will help synthesize, build and/or expand research foundations in the following focal areas: STEM learning, STEM learning environments, STEM workforce development, and broadening participation in STEM. The ECR program is distinguished by its emphasis on the accumulation of robust evidence to inform efforts to (a) understand, (b) build theory to explain, and (c) suggest interventions (and innovations) to address persistent challenges in STEM interest, education, learning, and participation.
In 2015, average mathematics scores on the National Assessment of Educational Progress (NAEP) declined in fourth and eighth grades, the first declines in mathematics at these grade levels since 1990. Declines in U.S. mathematics performance has important implications for overall STEM education as well as STEM workforce and international competitiveness. Researchers at Rutgers University will conduct an analysis to isolate the cause of the mathematics decline by investigating the dimensionality of the NAEP assessment, state-level outcomes, and demographic trends.
The team will use multilevel item response theory modeling techniques to investigate the declines by examining the factor structures to determine dimensionality across years. Researchers will examine subscores corresponding to each dimension of the factor structure at the state and national levels. In addition, subscores will be examined for trends in individual states and jurisdictions. Potentially, the analyses will allow for examination of factors related to state standards adoptions, demographic shifts, and participation rates.
DATE:
-
TEAM MEMBERS:
Gregory Camilli
resourceprojectProfessional Development, Conferences, and Networks
The Museum of Science (MOS) will conduct a conference and associated activities to consider ways to foster the STEM workforce via both science research and science communications experiences for students at the high school level. The work will draw on and expand the scope in the USA of the NSF-funded National Living Laboratory Network which currently involves more than 350 institutions across 48 states and 21 countries. The National Living Laboratory initiative involves university researchers and museum professionals in co-implementing research and science communications activities in museum settings with the public, primarily families with young children. The research and communications focus is on cognitive science pertaining to the development of young children. While many scientists and museum professionals are interested in integrating high school research experiences into their practices, particularly for under-represented youth, existing infrastructure at museums and universities limits the quality of experiences and quantity of students that institutions can support. Current academic and museum members of the National Living Laboratory community have identified an opportunity to advance shared interests and knowledge in engaging youth in STEM by leveraging the Living Laboratory framework.
This project involves pre- and post- conference activities and will convene a group of science research and museum professionals at a workshop in Boston, MA to: 1) document current opportunities and challenges in engaging high-school aged youth in cognitive science research activities; 2) outline strategies to engage youth in research and science communication through Living Laboratory, with particular emphasis on cognitive sciences; and 3) create and disseminate a report on workshop outcomes through existing communication channels in both fields. The project includes pre-conference surveys of professionals about the topic and an evaluation of the project activities and outcomes. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
InformalScience.org is an online collection of resources designed to serve a broad community of professionals whose work relates to informal education in science, technology, engineering, and math (STEM). Funded by the National Science Foundation and managed by the Center for Advancement of Informal Science Education (CAISE), InformalScience hosts a variety of usercontributed resources, including a wiki of evidence-based information on the impacts of informal STEM and a database of over 7,000 reports, articles, project descriptions, and other items uploaded by CAISE and website members.
The
While the term 'failure' brings to mind negative associations, there is a current focus on failure as a driver of innovation and development in many professional fields. It is also emerging from prior research that for STEM professionals and educators, failure plays an important role in designing and making to increase learning, persistence and other noncognitive skills such as self-efficacy and independence. By investigating how youth and educators attend to moments of failure, how they interpret what this means, and how they respond, we will be better able to understand the dynamics of each part of the experience. The research team will be working with youth from urban, suburban and rural settings, students from Title I schools or who qualify for free/reduced-price lunches, those from racial and ethnic minority groups, as well as students who are learning English as a second language. These youth are from groups traditionally underrepresented in STEM and in making, and research indicates they are more likely to experience negative outcomes when they experience failure.
The intellectual merit of this project centers on establishing a baseline understanding of how failure in making is triggered and experienced by youth, what role educators play in the process, and what can be done to increase persistence and learning, rather than failure being an end-state. The research team will investigate these issues through the use of qualitative and quantitative research methods. In particular, the team will design and evaluate the effectiveness of interventions on increasing the abilities of youth and educators in noticing and responding to failures and increasing positive (e.g., resilience) outcomes. Research sites are selected because they will allow collection of data on youth from a wide range of backgrounds. The research team will also work to test and revise their hypothesized model of the influence of factors on persistence through failures in making. This project is a part of NSF's Maker Dear Colleague Letter (DCL) portfolio (NSF 15-086), a collaborative investment of Directorates for Computer & Information Science & Engineering (CISE), Education and Human Resources (EHR) and Engineering (ENG).
This Research in Service to Practice project, a collaboration of Pepperdine University and the New York Hall of Science, will establish a network of STEM-related Media Making Clubs comprised of after-school students aged 12 - 19 and teachers in the U.S. and in three other countries: Kenya, Namibia and Finland. The media produced by the students may include a range of formats such as videos, short subject films, games, computer programs and specialized applications like interactive books. The content of the media produced by the students will focus on the illustration and teaching of STEM topics, where the shared media is intended to help other students become enthused about and learn the science. This proposal builds on the principal investigator's previous work on localized media clubs by now creating an international network in which after-school students and teachers will collaborate at a distance with other clubs. The central research questions for the project pertain to three themes at the intersection of learning, culture and collaboration: the impact of participatory teaching, virtual networks, and intercultural, global competence. The research will combine qualitative, cross-cultural and big data methods. Critical to the innovation of the project, the research team will also develop a network assessment tool, adapting epistemic network analysis methods to the needs of this initiative. This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
DATE:
-
TEAM MEMBERS:
Eric HamiltonKatherine McMillanPriya Mohabir
This summative evaluation report details the Broad Implementation of the Living Laboratory model--an initiative to promote partnership between museums and cognitive science researchers in order to promote professional learning and involve the public in scientific research. The evaluation investigated the extent of the dissemination effort’s depth, spread, sustainability, and shift in ownership, based on Coburn’s criteria for scale-up (2003). Evaluators collected data from surveys, interviews, focus groups, document review, and observations. Findings about depth suggest that adopters fully
Informal learning, and by extension, museums, are inherently emotional experiences, evoking feelings of awe, excitement, and curiosity. Oftentimes, museum professionals have prioritized traditionally positive emotions such as excitement and interest as being the most desirable and useful in supporting museum learning. However, prior research into naturally occurring emotions at museums found that some visitors who experienced negative emotions, such as confusion or frustration, at exhibits also reported deeper engagement and overall feelings of satisfaction (Rappolt-Schlichtmann et al., 2017). Based on these findings and similar results from formal education (D’Mello et al., 2014), this project team has worked to develop and refine a framework of strategies for creating exhibits that invoke and support visitors through the complex emotional state called productive struggle (PS) which is defined as a three-part emotional arc characterized by: 1) disequilibrium (experienced emotionally as emotions like confusion, frustration, surprise, or unease) that arises from encountering a challenging task, phenomenon, or idea, 2) persistence through the task which is supported by exhibit design scaffolds, and 3) an emotionally productive resolution tied to the source of disequilibrium or an overall sense of effortful achievement. In deliberately attending to and supporting a range of negative and positive emotions in museums, visitors can gain access to a wider variety of complex emotional experiences, including those critical to STEM learning, and have potential to broaden participation in STEM by supporting learners' diverse emotional needs and preferences.
This multidisciplinary project team consisting of researchers and exhibit professionals utilized a design-based research (DBR) process to develop, test, and refine a definition of museum-based productive struggle, and create a framework of design strategies that support PS. Three physical exhibits and a virtual exhibit were created during this multi-year project. Additionally, a summative research study was conducted with 105 youth ages 10-17 to explore: 1) whether visitors experienced the expected emotional arc of PS; 2) how exhibit design strategies supported PS; and 3) the extent of visitors’ learning and engagement at these PS exhibits.
This infographic presents a visual summary of a user study of the InformalScience.org website, which was conducted in 2016. The study used online surveys, task-based interviews, and web analytics to explore the following questions: (1) Who are the site's users and what content do they value? (2) Does the site clearly convey its purpose to visitors? and (3) Are users able to navigate the site and use search tools effectively?