This is the final evaluation report from RMC Research Corp. for the PES@LTERs project. Appendix includes instruments.
RMC Research designed evaluation activities to provide formative and summative feedback to Harvard Forest and the Hubbard Brook Research Foundation (Hubbard Brook) on their plan to embed public engagement with science (PES) into the cultures and practices of Long-Term Ecological Research Sites (LTERs) in the northeastern US. The purpose of this project was to build PES mechanisms into long-term ecosystem studies that create on-going, open exchanges between scientists and
Despite the fact that most science learning takes place outside of school, little is known about how engagement in informal science learning (ISL) experiences affects learners’ knowledge, skill development, interest, or identities over long periods of time. Although substantial ISL research has documented short-term outcomes such as the learning that takes place during a science center visit, research suggests that the genuine benefits of informal experiences are long-term transformations in learners as they pursue a “cascade” of experiences subsequent to the initial educational event
Public outdoor spaces present opportunities for social experiences and learning. This Broader Implementation project will expand and evaluate a model that transforms urban public spaces into accessible and engaging environments for learning social science in outdoor public spaces. The model combines social science inquiry exhibits, place making and human facilitation of learning experiences in outdoor public areas. Project exhibits use the facilitated social interactions as both the content of and medium for the experiences. This project will adapt the existing exhibits and add new exhibits and facilitation techniques for testing in three different urban environments. Project research will explore the efficacy of these adaptations and revised facilitation techniques for the different settings in collaboration with civic partners at each site. The project will share the model and research findings widely through the Exploratorium website and publications for researchers, developers, and educators.
The team’s prior research showed that facilitators improved multiple learning outcomes with the current exhibits. Visitors acquired new social observation skills, reflected on their own experiences, perceptions, and actions, and increased their awareness for how social behavior, cognition, and emotion can be studied scientifically. Building on the prior research, the project will install the exhibition and test its efficacy in three different urban environments and explore the adaptations that are required for different settings with different civic partners. The project will use design-based research to develop a new theoretical model of facilitation strategies for supporting science learning in outdoor public spaces. For evaluation, the project will use mixed methods, including observations, interviews, surveys, and document review. Evaluation will assess success in attracting and engaging visitors; conveying social science concepts; prompting self-reflection of judgments and actions; and fostering empathy among those with different social identities. The project will assess the extent to which participants, particularly those from marginalized communities, experience feelings of belonging and inclusion. The project will be presented in three sites which represent the significant diversity, income levels, and urban environments of San Francisco. Facilitation strategies are being co-developed with Urban Alchemy, an organization that works within distressed urban communities in San Francisco. Project site partners and collaborators include the San Francisco Public Library, the Port of San Francisco, and the San Francisco Department of Parks and Recreation. The project will also measure partnership outcomes, through surveys and interviews, to look at the extent and ways the project integrates a co-creation model and develops an authentic, mutually beneficial, sustainable partnership. The project will generate and disseminate generalizable knowledge about the affordances of combining informal science learning, placemaking, and facilitation in a variety of free, outdoor STEM learning spaces in collaboration with local community groups. The project will also advance public understanding of the social and behavioral sciences.
This research project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to (a) advance new approaches to and evidence-based understanding of the design and development of STEM learning in informal environments; (b) provide multiple pathways for broadening access to and engagement in STEM learning experiences; (c) advance innovative research on and sssessment of STEM learning in informal environments; and (d) engage the public of all ages in learning STEM in informal environments.
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
Zoos and aquariums have been offering programming, events, and visit accommodations to autistic individuals for several years. While these efforts can provide great experiences, they are focused more on accommodation and the outward-facing guest experience than inclusion. Lack of inclusion features in design, programming, and representation amongst zoo and aquarium representatives, ultimately limits full inclusion and adds to a sense in autistic individuals of not belonging and not being welcomed. To develop a fully inclusive experience for autistic individuals, this project will develop an evidence-based framework of inclusive practices for zoos and aquariums and build a community of practice around inclusion broadly. The project brings together researchers from Oregon State University, Vanderbilt Kennedy Center’s Treatment and Research Institute for Autism Spectrum Disorders, and the Association of Zoos and Aquariums. Researchers will create and investigate the extent and ways in which a research-informed framework and associated tools (i.e. case studies, discussion guides, self-guided audits, etc.) and strategies support science learning for autistic individuals, and help practitioners expand access and inclusion of autistic audiences beyond special events or the general visit experience by applying inclusive practices for programs, exhibit development, internships, volunteer opportunities, and employment. To maximize impact, the project will develop and expand a network of early adopters to build a community of practice around inclusive practices to develop fully inclusive zoo and aquarium experiences for all individuals.
The project will investigate 4 research questions: (1) In what ways and to what extent are zoos and aquariums currently addressing access and inclusion for autistic individuals? (2) How do staff in zoos and aquariums perceive their and their institution’s willingness and ability to address access and inclusion for autistic individuals? (3) What is a framework of evidence-based practices across the zoo and aquarium experience that is inclusive for autistic individuals, and what associated tools and strategies are needed to make the framework useful for early adopters? And (4) to what extent and in what ways does a research informed framework with associated tools and strategies engage, support, and enhance an existing community of practitioners already dedicated to addressing autistic audiences and promote inclusive practices at zoos and aquariums for autistic people? The project is designed as two phases: (1) the research and development of a framework of inclusive practices and tools for supporting autistic individuals and (2) expanding a network of early adopters to build a community of practice around inclusive practices and an overall strategy of implementation. The framework will be informed through a state of the field study across the zoo/aquarium field that includes a landscape study and needs assessment as well as a review of literature that synthesizes existing research across disciplines for developing inclusive practices for autistic individuals in zoos and aquariums. The team will also conduct online surveys and focus groups to gather input from various stakeholders including zoo and aquarium employees and practitioners, autistic individuals, and their social groups (e.g., family members, peers, advocacy organizations). The second phase of the study will focus on sharing the framework and tools with practitioners across the zoo/aquarium field for feedback and reflection to develop an overall strategy for broader implementation and expanding the existing network of zoo and aquarium professionals to build a community of practice dedicated to the comprehensive inclusion of autistic individuals across the full zoo and aquarium experience. The results will be disseminated through conference presentations, scholarly publications, online discussion forums, and collaborative partners’ websites. The project represents one of the first of its kind on autistic audiences within the zoo and aquarium context and is the first to look at the full experience of autistic patrons to zoos and aquariums across programs/events, exhibits, volunteering, internship, and employment opportunities. A process evaluation conducted as part of the project will explore how the approach taken in this project may be more broadly applied in understanding and advancing inclusion for other audiences historically underserved or marginalized by zoos and aquariums.
This Research in Service to Practice project is supported by the Advancing Informal STEM Learning (AISL) program.
This project investigates long-term human-robot interaction outside of controlled laboratory settings to better understand how the introduction of robots and the development of socially-aware behaviors work to transform the spaces of everyday life, including how spaces are planned and managed, used, and experienced. Focusing on tour-guiding robots in two museums, the research will produce nuanced insights into the challenges and opportunities that arise as social robots are integrated into new spaces to better inform future design, planning, and decision-making. It brings together researchers from human geography, robotics, and art to think beyond disciplinary boundaries about the possible futures of human-robot co-existence, sociality, and collaboration. Broader impacts of the project will include increased accessibility and engagement at two partner museums, interdisciplinary research opportunities for both undergraduate and graduate students, a short video series about the current state of robotic technology to be offered as a free educational resource, and public art exhibitions reflecting on human-robot interactions. This project will be of interest to scholars of Science and Technology Studies, Human Robotics Interaction (HRI), and human geography as well as museum administrators, educators and the general public.
This interdisciplinary project brings together Science and Technology Studies, Human Robotics Interaction (HRI), and human geography to explore the production of social space through emerging forms of HRI. The project broadly asks: How does the deployment of social robots influence the production of social space—including the functions, meanings, practices, and experiences of particular spaces? The project is based on long-term ethnographic observation of the development and deployment of tour-guiding robots in an art museum and an earth science museum. A social roboticist will develop a socially-aware navigation system to add nuance to the robots’ socio-spatial behavior. A digital artist will produce digital representations of the interactions that take place in the museum, using the robot’s own sensor data and other forms of motion capture. A human geographer will conduct interviews with museum visitors and staff as well as ethnographic observation of the tour-guiding robots and of the roboticists as they develop the navigation system. They will produce an ethnographic analysis of the robots’ roles in the organization of the museums, everyday practices of museum staff and visitors, and the differential experiences of the museum space. The intellectual merits of the project consist of contributions at the intersections of STS, robotics, and human geography examining the value of ethnographic research for HRI, the development of socially-aware navigation systems, the value of a socio-spatial analytic for understanding emerging forms of robotics, and the role of robots within evolving digital geographies.
This project is jointly funded by the Science and Technology Studies program in SBE and Advancing Informal STEM Learning (AISL) Program in EHR.
Recent studies have advocated for a shift toward educational practices that involve learners in actively contributing to science, technology, engineering, and mathematics (STEM) as a shared and public endeavor, rather than limiting their involvement to the construction of previously established knowledge. Prioritizing learners’ agency in deciding what is worth knowing and how learning takes place may create more equitable and inclusive learning experiences by centering the knowledge, cultural practices, and social interactions that motivate learning for people across ages, genders, and backgrounds. In informal learning environments, families’ social interactions are critical avenues for STEM learning, and science centers and museums have developed strategies for prompting families’ sustained engagement and conversation at STEM exhibits. However, exhibits often guide visitors’ exploration toward predetermined insights, constraining the ways that families can interact with STEM content, and neglecting opportunities to tap into their prior knowledge. Practices in the maker movement that emphasize skill-building and creative expression, and participatory practices in museums that invite visitors to contribute to exhibits in consequential ways both have the potential to reframe STEM learning as an ongoing, social process that welcomes diverse perspectives. Yet little is known about how these practices can be scaled, and how families themselves respond to these efforts, particularly for the diverse family audiences that science centers and museums aim to serve. Further, although gender and ethnicity both affect learning in informal settings, studies often separate participants along a single dimension, obscuring important nuances in families’ experiences. By addressing these outstanding questions, this research responds to the goals of the Advancing Informal STEM Learning (AISL) program, which seeks to advance evidence-based understanding of the design and development of STEM learning opportunities for the public in informal environments. This includes providing multiple pathways for broadening engagement in STEM learning experiences and advancing innovative research on STEM learning in informal environments.
Research will address (1) how families perceive and act on their collective epistemic agency while exploring STEM exhibits (i.e., how they work together to negotiate and pursue their own learning goals); (2) whether and how families’ expressions of agency are influenced by gender and ethnicity; and (3) what exhibit design features support expressions of agency for the broadest possible audience. Research studies will use interviews and observational case studies at a range of exhibits with distinct affordances to examine families’ epistemic agency as a shared, social practice. Cultural historical activity theory and intersectional approaches will guide qualitative analyses of families’ activities as systems that are mediated by the physical environment and social setting. Education activities will involve an ongoing collaboration between researchers, exhibit designers, educators, and facilitators (high-school and college-level floor staff), using a Change Laboratory model. The group will use emerging findings from the research to create a reflection tool to guide the development of more inclusive learning experiences at STEM exhibits, and a set of design principles for supporting families’ expressions of agency. A longitudinal ethnographic study will document the development of inclusive exhibit design practices throughout the project as well as how the Change Lab participants develop their sociocultural perspectives on learning and exhibit design over time. Analyzing these shifts in practice within the Change Lab will provide a deeper understanding of what works and what is difficult or does not occur when working toward infrastructure change in museums. By considering how multiple aspects of families’ identities shape their learning experiences, this work will generate evidence-based recommendations to help science centers and museums develop more inclusive practices that foster a sense of ownership over the learning process for the broadest possible audience of families.
This Pilot and Feasibility study will build foundational knowledge about basic aspects of STEM webcams in the United States (US) from the perspectives of both practitioners and viewers. Thousands of webcams available to the public are operated by STEM organizations, such as zoos, museums, and government agencies. Learning theory suggests that STEM webcams, especially those with accompanying interpretive tools, have the potential to offer rich informal learning opportunities. However, yet no research has quantified any aspect (cognitive, behavioral, or emotional) of viewer outcomes. This study will be the first to develop baseline data regarding cognitive, behavioral, and emotional aspects of perceived viewer experience. Project activities include:
An inventory of STEM webcams that exist in the US, the STEM disciplines they represent, learning and engagement tools they employ, the number of viewers they reach, and the resources required for their operation
A survey of webcam operators, their STEM education goals, implementation strategies, and evaluation results; and
Surveys and interviews gathering data on viewers demographics and potential increase in curiosity, interest, knowledge, and behavior toward the STEM subject. This research will provide foundational knowledge for the STEM-education and research community that quantifies and describes many facets of the population of STEM webcams in the inventory.
Research activities will take place in three distinct phases, with Phase 1 laying the groundwork for Phases 2 and 3. Phase 1: The project team will conduct a systematic internet search for all identifiable STEM related webcams. Phase 2 (operator-focused): An online survey of practitioners of webcams operated by US-based STEM organizations will be conducted using Qualtrics software. Likert scales will be used. Various hypotheses will be tested regarding webcam program objectives, operations, and evaluations from the perspective of program operators or practitioners. Phase 3 (viewer-focused): Surveys and interviews with likely viewers of STEM webcams. Using the webcam inventory built in Phase 1, the team will collaborate with 20 informal STEM institutions that agree to survey their constituents to test hypotheses regarding webcam viewing practices, such as why and how viewers watch, and perceived outcomes of viewing, such as perceived influence on their interest, attitudes, knowledge, or behavior. The findings from this study will be widely shared with informal STEM institutions and webcam operators. It will provide foundational data for future experimental studies.
As science communication programs grow worldwide, effective evaluation and assessment metrics lag. While there is no consensus on evaluation protocols specifically for science communication training, there is agreement on elements of effective training: listening, empathy, and knowing your audience — core tenets of improvisation. We designed an evaluation protocol, tested over three years, based on validated and newly developed scales for an improvisation-based communication training at the Alan Alda Center for Communicating Science. Initial results suggest that ‘knowing your audience’ should
DATE:
TEAM MEMBERS:
Christine O’ConnellMerryn McKinnonJordan Labouff
Millions of people around the world watch live streaming wildlife cams, but they aren’t just watching: they are asking questions, trading information, and witnessing events that may be undocumented in the scientific literature. The goal of Bird Cams Lab was to design a digital space and framework enabling online communities to engage in a co-created scientific inquiry process utilizing wildlife cams to answer bird-related questions of common interest. To achieve this goal, the project engaged participants at every stage of the research process—including observation, generating and selecting
Student engagement is an important predictor of choosing science-related careers and establishing a scientifically literate society: and, worryingly, it is on the decline internationally. Conceptions of science are strongly affected by school experience, so one strategy is to bring successful science communication strategies to the classroom. Through a project creating short science films on mobile devices, students' engagement greatly increased through collaborative learning and the storytelling process. Teachers were also able to achieve cross-curricular goals between science, technology
DATE:
TEAM MEMBERS:
Kaitlyn MartinLloyd DavisSusan Sandretto