The National Building Museum (NBM) contracted RK&A to evaluate Creative-in-Residence (CIR), a program that invites visual and performing artists to NBM for short-term residencies to create original work that promotes engagement with the built environment. The study goal was to consider future implications for the CIR program based on the most recent CIR iteration (a January 2019 dance performance inviting visitors to explore NBM’s historic building) and past program iterations.
How did we approach this study?
To hear a variety of perspectives on CIR, RK&A conducted in-depth telephone
The purpose of the Science Center Public Forums project was to engage citizens with NOAA data about climate-related hazards, resilience strategies, and related policies. Forum modules about four climate-related hazards were created, and used as a part of forum programs at eight museums around the US. Evaluation findings are structured around three themes: 1) learning, 2) interest, engagement, and attitudes, and 3) educator outcomes. Data showed high levels of prior knowledge about environmental hazards and interactions between human and natural systems; resilience efforts; and the ways science
This project will examine the characteristics and outcomes of a large sample of environmental education field trip programs for youth to elucidate program characteristics that most powerfully influence 21st century learning outcomes. Environmental education programs for youth, particularly day-long school trip programs, are popular and reside at the intersection of formal and informal STEM education. Such field trips provide opportunities for diverse audiences to participate in shared learning experiences, but current understanding of what leads to success in these programs is limited. This large-scale study will address this gap in knowledge by investigating the linkages between program characteristics and participant outcomes for at least 800 single-day environmental education field trip programs for youth in grades 5-8, particularly programs for diverse and underserved audiences. This study will result in the identification of evidence-based practices that will inform future program design for a wide variety of settings, including nature centers, national parks, zoos, museums, aquaria, and other locations providing informal environmental education programs.
This Research in Service to Practice study is guided by two research questions: 1) What program characteristics (context, design, and delivery) most powerfully influence learner self-determination and learner outcomes? And 2) Do the most influential program characteristics differ across diverse and underserved audiences (e.g. African American, Hispanic/Latino, economically disadvantaged) and contexts (e.g. rural versus urban)? This project will examine a wide range of program-related factors, including pedagogical approaches and contextual characteristics. A valid and reliable protocol for observing 78 program characteristics hypothesized to influence learner outcomes developed by a previous project will be used to systematically sample and observe 500 single-day environmental education field trip programs for youth in grades 5-8 distributed across at least 40 U.S. states and territories. Programs for diverse and underserved youth will be emphasized, and a diverse set of programs in terms of program type and context will be sought. Data from this sample will be combined with those of an existing sample of 334 programs provided by over 90 providers. The final combined sample of over 800 programs will provide sufficient statistical power to confidently identify which program components are most consistently linked with learning outcomes. This sample size will also enable stratification of the sample for examination of these relationships within relevant subpopulations. Principal component analyses will be used to reduce data in theoretically meaningful and statistically valid ways, and multilevel structural equation modeling will be employed to examine the influences of both participants' individual characteristics and program and context characteristics on participant outcomes. Since one research question focuses on whether program outcomes are the same across different audiences, the project will include at least 200 programs for each of three specific audiences to ensure sufficient statistical power for confidence in the results: primarily African American, primarily Hispanic/Latino, and primarily White.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Robert PowellMarc SternBrandon Frensley
This three-year research and implementation project empowers middle school LatinX youth to employ their own assets and funds of knowledge to solve community problems through engineering. Only 7% of adults in the STEM job cluster are of Hispanic/Latino origin. There is a continuing need for filling engineering jobs in our current and future economy. This project will significantly broaden participation of LatinX youth in engineering activities at a critical point as they make career decisions. Design Squad Global LatinX expands on a tested model previously funded by NSF and shown to be successful. It will enable LatinX youth to view themselves as designers and engineers and to build from their strengths to expand their skills and participation in science and engineering. The project goals are to: 1) develop an innovative inclusive approach to informal engineering education for LatinX students that can broaden their engineering participation and that of other underrepresented groups, (2) to galvanize collaborations across diverse local, national, and international stakeholders to create a STEM learning ecosystem and (3) to advance knowledge about a STEM pedagogy that bridges personal-cultural identity and experience with engineering knowledge and skills. Project deliverables include a conceptual framework for a strength-based approach to engineering education for LatinX youth, a program model that is asset based, a collection of educational resources including a club guide for how to scaffold culturally responsive engineering challenge activities, an online training course for club leaders, and a mentoring strategy for university engineering students working with middle school youth. Project partners include the global education organization, iEARN, the Society of Women Engineers, and various University engineering programs.
The research study will employ an experimental study design to evaluate the impact on youth participating in the Design Squad LatinX programs. The key research questions are (1) Does participation increase students' positive perceptions of themselves and understanding of engineering and global perspectives? (2) To what extent do changes in understanding engineering vary by community (site) and by student characteristics (age, gender, ethnicity)? (3) Do educators and club leaders increase their positive perceptions of youths' funds of knowledge and their own understanding of engineering? and (4) Do university mentors increase their ability to lead informal engineering/STEM education with middle school youth? A sample from 72 local Design Squad LatinX clubs with an enrollment of 10-15 students will be drawn with half randomly assigned to the participant condition and half to the control condition. Methods used include pre and post surveys, implementation logs for checks on program implementation, site visits to carry out observations, focus groups with students and interviews with adult leaders. Data will be analyzed by estimating hierarchical linear models with observations. In addition, in-situ ethnographically-oriented observations as well as interviews at two sites will be used to develop qualitative case studies.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Mary Haggerty
resourceprojectProfessional Development, Conferences, and Networks
This project will advance evidence-based efforts to broaden informal STEM engagement via the 2019 Inclusive Science Communication (Inclusive SciComm) Symposium, to be held September 27-29, 2019, at the University of Rhode Island in Kingston, Rhode Island. Science communication, defined as any information exchange designed to engage targeted audiences in conversations or activities related to STEM topics, is a rapidly expanding area of research and practice with the potential to significantly increase public participation and sense of belonging in STEM fields. That said, there are few opportunities for its practitioners and scholars to convene around how to make their work both inclusive and equitable, which collectively acknowledge identity, cultural differences, and epistemologies as part of broadening participation. The 2019 symposium will address this gap through panels, workshops, and posters focused on three themes that represent critical and difficult aspects of inclusive science communication: (1) New Languages, Practices, Knowledge, and Research; (2) Changing Systems and Structures through Science Communication; and (3) Social Responsibility and Ethics. Within these themes, sessions will be organized to address major barriers of absence identified by participants in the 2018 Inclusive SciComm Symposium: skills, lessons learned, and knowledge gaps, especially with regard to facilitating difficult conversations across difference (critical dialogue). The symposium also will emphasize the need to integrate research and practice to advance inclusive, equitable, and intersectional approaches to science communication.
There is an urgent need to question assumptions and examine evidence regarding how science communicators and scholars approach efforts to broaden participation, but insufficient data exist on the inputs and outputs of inclusive and equitable practice. Critical dialogue about potentially uncomfortable topics such as privilege, power, or marginalization is an essential tool for inclusive practice and pedagogy. Finding from the 2018 Inclusive SciComm Symposium indicated that many educators and practitioners lack the language, skills, or confidence to initiate this type of dialogue. This project supports the knowledge-building component of the 2019 Inclusive SciComm Symposium to inform future science communication training, practice, and scholarship, by building on preliminary data collected during the 2018 symposium and responding to the need for more robust evaluation of science communication activities. Applying the Theory of Planned Behavior, the project will employ pre/post symposium surveys to investigate how 2019 symposium activities affected knowledge, attitudes, subjective norms, and efficacy (the variables of the Theory of Planned Behavior) of attendees with regard to critical dialogue. Focus groups at the symposium will be used to identify priority research areas related to inclusion, generally, and critical dialogue, specifically, that could advance inclusive science communication practice and beneficial outcomes. The project also will evaluate symposium impacts with regard to 1) attendees' opinions on utility of symposium components for advancing inclusive science communication and 2) how attendees' experience and response orientations inform their approaches to difficult science communication conversations. Qualitative data from the surveys and focus groups will be thematically coded using constant comparison.
This project will have strategic impact for inclusive science communication practice and, therefore, for informal learning and public engagement with STEM topics. Increasing awareness and effective implementation of critical dialogue by science communicators and trainers should enhance both for ethical engagement of traditionally under-represented and marginalized groups and should foster diverse types of public participation in societal debates about scientific issues. The outcomes of this research will benefit and link the complementary, but often siloed, fields of informal science learning and science communication. A final report will summarize research findings and offer specific next steps to advance inclusive science communication practice and research, especially with regard to fostering critical dialogue. The report will be posted on inclusivescicomm.org and distributed via a national network of partners working in informal science education, science communication, and public engagement.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Recognizing that race can influence African American youths' perception of which academic disciplines and careers are available to them, this pilot study will explore how African American youths' physical and social communities can be leveraged to support the evolution of their STEM identity and their ability to recognize their potential as scientists. Unfortunately, many of these youths live in communities that are void of critical resources that research has demonstrated time and time again are critical for success in STEM disciplines and careers. This lived reality for many African American youth is the direct result of long-standing disparities in access and opportunities, fueled by racial socialization and biased institutional structures. This pilot will empower youth to recognize these disparities and use science to provide solutions. One perilous societal disparity experienced in many predominately African American communities is the lack of access to fresh produce and healthy food. As a mechanism for potential resolution, this project will consider the utility of community gardens to address this important community need and as a strategy to engage youth in STEM content and skill development. While this notion is not novel to NSF, the intent to utilize an augmented reality (AR) storytelling platform for data collection and project experiences is innovative. This technology will also provide a space for participants to share their work with each other and their broader communities. To our knowledge, this pioneering approach has not been previously piloted in this context. In addition, the pilot will engage multiple youth serving community-based organizations such as park and recreation centers and faith-based organizations in this work, which is also innovative. This is significant, as youth serving community-based organizations are often play important role in the social, educational, and cultural lives of youth and their families in communities. These organizations are often at the heart of the community, figuratively and literally. If successful, this pilot could be transformative and provide a strong basis to support similar work in other communities.
Over the two-year project duration, eighty African American youth ages 11 -14 will participate in the year-long program, across three youth-serving, community-based organizations at four sites. They will be exposed to relevant agricultural, geological, engineering and technological content through a newly developed curriculum called "Cultivating My Curriculum." Community mentors and undergraduate role models will facilitate the instruction and hands-on experiences in the garden and with the AR platform. A capstone event will be a held for the participants and community to convene to learn more about the results of the pilot and share recommendations with community leaders for improving the disparities identified during the pilot. The research component will focus on: (a) the impact of the sociocultural theoretical framework grounding the work on youths' STEM identities, (b) the integration of the AR tool, and (c) mentorship. Formative and summative evaluation will take place through focus groups, surveys, journals, and youth storytelling. Ultimately, the project endeavors to advance the narrative that African Americans are scientists and that science can be used to improve the lives of African Americans and other groups challenged by structural and racial disparities.
This pilot study is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Harrison PinckneyDavid BoyerBarry GarstDilrukshi Thavarajah
resourceprojectProfessional Development, Conferences, and Networks
STEAM, the use of art as a context and tool for science education, is currently a hot topic in the science education field. In almost all instances of study and practice, it involves the use of science-themed or science-informed art in science education. As such, it does not take advantage of the majority of artistic output that does not have an obvious connection to science. The National Academies of Sciences, Engineering and Medicine recently called for more research to expand the "limited but promising" evidence that integrating arts and humanities with science education leads to better learning. The goal of this 2.5-day conference is to bring together representatives of both art and science groups to have a shared discussion around how non-scientific art can influence science education in theory, and how we can apply empirical results to the theory. For purposes of this conference, "non-science art" is defined as art that was not inspired by science. Conference attendees will include researchers (art and science education researchers) and practitioners (artists, art museum interpreters, and science educators). The conference will take place during the 2020 Black Creativity exhibition at the Museum of Science and Industry, Chicago. It is anticipated that by holding the conference at that time the audience for the conference and its impact will be informed by more diverse attendance.
The conference will be implemented starting with a pre-conference reading. Attendees will be sent a copy of the white paper from the Art as a Way of Knowing report for background reading and also asked to contribute to a Google Document that describes their various contexts. Each day of the conference will focus on a theme -- state of the field and possibilities and research -- and be comprised of large and small group interactions. Attendees will be invited from the ranks of practitioners, researchers and educators in the art and science education fields; several slots will be available for open (non-invited) participants. Key outcomes include: (a) a summary of all the research that has been conducted on using non-science art in science education, (b) starting points for building a theory on why non-science art can be used in science education; and (c) a list of specific research topics that would help inform, advance, and test the theory. In addition to assessing satisfaction with the conference, evaluation will also include a one-year post conference survey to investigate impact of participation in the conference.
This conference will generate products that will give guidance to both researchers and practitioners who want to use art in science education. These products include a white paper synthesizing the discussion and appendices that include raw transcripts and a bibliography of resources. Another product is a roadmap to create interventions that can be studied, which should lead to a stronger, more rigorous theory of practice about how art can be integrated into science education.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Many of the Hispanic children and families who live in the Rio Grande Valley lack opportunities to engage in inspirational and educational experiences introducing Science, Technology, Engineering and Mathematics (STEM) concepts and related careers. The University of Texas, Rio Grande Valley (UTRGV) will adapt and research the "Energy and U Show," which will introduce thousands of children and families to an exciting and dramatic that shows interconverting different forms of energy. The show will meld the excitement of chemical demonstrations and the natural connection between energy and STEM education in a fully produced, on-stage science extravaganza. A foundational philosophy of the show is that there is additional real value in getting children and youth onto a college campus. For many of its participants, this is their first time sitting in a seat at a university, the first opportunity for them to envision themselves in this environment. In partnership with the University of Minnesota, which originally developed the show, UTRGV will adapt the show, now presented in English, to a bilingual, culturally accessible format that is designed to Hispanic family audiences and student groups in learning about energy and related careers. Evaluation results demonstrate that the show has effectively engaged thousands of Minnesota students. The target audience will be upper elementary (4th-5th grade), middle school students, and their parents. This project will be led by UTRGV, nation's second-largest Hispanic Serving Institution, with a student enrollment of 28,000, of which over 90% are Hispanic and more than 60% are first-generation college students). In addition to the show, the project will include: (1) a manual to guide implementation of the program and related resources at different national or international venues; (2) educational resources for parents, teachers and school counselors introducing STEM careers and specific STEM college majors; (3) mentoring of UTRGV faculty in outreach activities; and (4) dissemination of the show to other campuses and venues.
The project will conduct ongoing research and evaluation guiding the adaptation of the show and investigation of factors contributing to positive educational impacts of the project, which will be carried out by a bilingual/bicultural researcher. Project research instruments will measure student level of engagement, interest and learning, as well as college interest, in surveys and analysis of data pre and post demonstration. The project will specifically investigate the impact of language on student impacts. Each component of this project will be studied to determine program intervention effectiveness (the scientific demonstration and language of the demonstration). To determine program effectiveness, a baseline of data before program implementation will be established concerning Hispanic students, their persistence, and perceptions of the environment. The project will measure parent perceptions of STEM careers for their children through pre and post demonstration surveys and focus groups. Student and parent research participants will be able to use surveys or respond to other research activities in the language of their choice. Project findings will contribute to the knowledge base concerning how linguistically and culturally adapted science shows and related resources adapted into can have positive impacts regarding the STEM knowledge and careers of students and parents from low-income and Hispanic communities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Karen LozanoArturo FuentesAaron MassariBrian Warren
resourceevaluationProfessional Development, Conferences, and Networks
This report summarizes the evaluation outcomes of the Collaborative Project Management (CoPM) Institute project, a two-day convening centered on bringing proven project management skills and approaches in the for-profit world to the informal science education (ISE) community to support effective collaborations and successful outcomes. The CoPM Institute was conceptualized as a pilot effort to test the effectiveness of porting of tools and frameworks for collaborative project management from the business/for-profit environment to advance the following project goals:
Build the capacity of
Informal learning institutions, such as science centers and museums, are well-positioned to broaden participation in engineering pathways by providing children from underrepresented groups with motivational, self-directed engineering design experiences. Though many informal learning institutions offer opportunities for young visitors to engage in engineering activities, little is known about the specific features of these activities that support children's motivation in engineering design processes such as problem scoping, testing, and iteration. This project will address this gap and advance foundational knowledge by identifying features of engineering design activities, as implemented within an informal setting, which support underrepresented children's engineering motivation and persistence in engineering tasks. Researchers at New York Hall of Science (NYSCI) will observe children interacting with families and museum educators as they engage in different engineering design activities in NYSCI's Design Lab, an exhibition space devoted to hands-on exploration of engineering design. They will also survey and interview the children and their caregivers about these experiences. Analyses of these data sources will result in a description of features of design activities foster motivation and task persistence in engineering design. Findings will be disseminated nationally to other informal learning institutions, which in turn can use the knowledge generated from this project to create motivational, research-based, field-tested engineering design experiences for young visitors, especially for children from underrepresented groups. The experiences may encourage children to further pursue engineering pathways, resulting in a diversified engineering workforce with the potential to drive and sustain national innovation and global technological leadership.
This project uses the framework of goal orientation, defined as learners' self-reflection of why and how they engage in tasks, to understand whether, how, and why underrepresented 7-12-year-olds engage in engineering design activities in an informal learning institution. Though previous research has suggested that goal orientation is strongly, positively related to learning and motivation in formal settings such as schools, research in informal settings has not robustly accounted for the role of goal orientation in participants' engagement with learning tasks in these unique learning environments. To better understand how children's goal orientations contribute to their motivation in engineering in informal learning institutions, researchers will answer the following research questions: (1) What are underrepresented children's goals and goal orientations while participating in engineering design activities in an informal setting? (2) What contextual factors--including facilitation strategies, materials, task relevance, and social interactions with family members--may support or discourage the adoption of different goal orientations? (3) How do goal orientations relate to children's learning experience in the engineering design activities and the likelihood that they will test and iterate their solutions? These questions will be answered through a mixed-method research study conducted with approximately 200 families, with children aged 7-12, recruited from underrepresented groups. Semi-structured clinical interviews, conducted with 20% of the children and their caregivers, as well as observations and surveys gathered from all families, will provide information on the children's goal orientation and engagement as they relate to specific engineering design activities. Qualitative content analyses and multilevel structural equation modeling will result in findings that will be disseminated widely to other institutions of informal learning. Ultimately, this project will generate new empirical knowledge regarding the features of engineering design activities in informal learning environments that increase engineering engagement and motivation among underrepresented children, thereby broadening participation in engineering pathways.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
There are several critical reasons to understand and support interest development in early childhood: (a) as a primary motivator of engagement and learning; (b) interest development in preschool predicts important learning outcomes and behaviors in early elementary school; and (c) early childhood interests motivate ongoing interest development. Thus, there is growing recognition that interest is not just important but fundamental to education and learning. Head Start on Engineering (HSE) is a multi-component, bilingual (Spanish/English), family-focused program designed to (1) foster long-term interest in the engineering design process for families with preschool children from low-income backgrounds and (2) support family development and kindergarten readiness goals. The HSE program, co-developed with the Head Start community, provides families with developmentally appropriate, story-based engineering design challenges for the home and then connects these to a system of strategically aligned Informal STEM Education (ISE) experiences and resources. This current project, HSE Systems, builds on a previous HSE Pathways project which (a) established that participating families develop persistent engineering-related interests; (b) highlighted the value that the Head Start community has for the program and partnership; and (c) generated a novel, systems perspective on early childhood interest development. The aim of HSE Systems is to develop and test a model of early childhood STEM engagement and advance knowledge of how the family as a system develops interest in STEM from preschool into kindergarten.
Through the Design Based Implementation Research (DBIR) process, the team will iteratively refine and improve the HSE program and theory of change using ongoing feedback and data from staff, families, and partners. It is also designed to explore program impacts on family interest development over a longer period, as children enter kindergarten. The DBIR work will focus primarily on the program model questions, while the case study research will focus on the family interest questions, with both strands informing each other. The initial work is organized around a series of feedback and design-testing cycles to gather input from families and other stakeholders, update the program components and activities in collaboration with families and staff, and prepare for full implementation. During the next phase, the team will implement the full program model with six Head Start classrooms and track family experiences and interest development into kindergarten. During final implementation phase, the team will finish data collection, conduct retrospective analysis with all the data, and update the program model and theory of change.
This project will directly address the AISL program goals by broadening access to early childhood informal STEM education for low-income communities, with a focus on Spanish-speaking families, and building long-term skills and learning dispositions to support STEM learning inside and outside of school. Beyond the topic of engineering, HSE supports Head Start school readiness and child and family development goals, which are the foundation of lifelong success.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This project in the Advancing Informal STEM (Science, Technology, Engineering and Mathematics) Learning program's Innovations in Development track aims to build professional capacity in Informal Science Education (ISE) institutions for effective engagement of Latinx audiences. A collaboration between The Exploratorium and the Children's Discovery Museum (CDM) of San Jose, Cambio is a professional development project based on the premise that developing cultural competence specific to Latinx communities and STEM learning, together with organizational change capacity, will enable ISE institutions to improve their ability to be inclusive of Latinx communities, cultures, and audiences. Cambio's ultimate aim is to broaden participation in STEM by building the ISE field's capacity to effectively engage Latinxs in informal STEM learning. ISE institutions and other out-of-school programs and organizations have an important role to play in inspiring and preparing the next generation of Latinx STEM students, employees, and educators. Cambio participants will deepen their engagement with the research and practice relevant to this role and build an understanding of what broadening Latinx engagement can and should look like in their institutions. Working with expert instructors and coaches, and together as peers, Cambio participants will apply what they learn to real-world strategic initiatives they implement at their home institutions.
The heart of the Cambio project is the creation of a professional development (PD) model for ISE institutions and their staff that synthesizes: (a) current knowledge about strategies for Latinx engagement in informal STEM learning; (b) previous NSF-funded projects that leveraged that knowledge to chart a way forward for the field; and (c) CDM's Cultural Competence Learning Institute professional development program for building cultural competence, inclusion, and organizational change in museums. This synthesis will form a robust professional development platform that has the potential to create a field-wide shift in the way informal science institutions approach working with Latinx audiences. The Cambio professional development program will include: a new professional development framework and curriculum that will reach 54 practitioners in 15 institutions; the development and dissemination of professional development tools and resources for use by ISE practitioners; a Community of Practice focused on Latinx engagement in informal STEM learning; evidence of the efficacy of the Cambio PD model; and knowledge generated by formative and summative evaluation that will inform other ISE efforts focused on increasing the participation of Latinxs in STEM. A STEM focus will be woven throughout the professional development experience that focuses explicitly on areas of intersection between Latinx culture and identities and STEM. Practitioners will increase their expertise in designing experiences that will invoke emotional engagement, spark curiosity and excitement, in ways that explicitly value Latinx identities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.