The summative evaluation documents and articulates what SCIENCES has improved or changed, and in what ways. The final design of the summative evaluation was based on findings from the front-end and formative evaluations, including using participatory evaluation techniques to engage community members in discussing their experience with the programs and assessment of community needs and assets at the close of the project.
The goal of the summative evaluation was to address discrete program impacts in the context of the project, as well as the cross-program impact of providing a thematically
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The theme of this conference project by the New York Hall of Science will be exploring how to better design exhibits to promote "public engagement with science." Here, "public engagement with science" refers to opportunities that go beyond traditional approaches to the public understanding of science. The event will invite professionals to consider how to shift exhibit designs toward engaging visitors with STEM in ways that emphasize the intersection of STEM innovation with visitors' daily lives, their personal agency, and their interdependence with their personal social networks and the institutions that advance STEM knowledge and innovation. The conference and its pre- and post-conference activities will bring together curators, exhibition developers, community outreach professionals, museum administrators, and learning scientists from the United States and Canada. They will work together to identify design principles and key obstacles to designing exhibits that can better help science museums achieve two goals: 1) making visitors' diverse and personal questions, concerns, and perspectives central to their experience of the exhibits; and 2) engaging visitors as contributors to the exhibit experience in ways that make their contributions visible and consequential. During this two-day event attendees will consider how exhibits can support broader and more diverse public participation in critical debates about the roles of STEM discovery and innovation in society. The effort is grounded in recent work on public engagement with science; on reorganizing museums to become sites for participation and contribution by visitors, and particularly by institutions' local communities; and on making and engineering design programming within museums. The goal is to chart a course toward a vision of the future of science museums in which they maintain their status as sources of trusted information, while also fulfilling their potential as sites of genuine participation and social interaction, in which visitors make meaningful contributions to the substance and workings of the museum floor while also engaging with, learning about and holding themselves accountable to the core concepts and practices of the STEM disciplines. The conference will build the capacity and collaborative engagement of a network of science centers whose work is central to achieving the museum field's ultimate goal of engaging the public of all ages in learning STEM in informal environments. The conference and associated activities will be evaluated by staff at the New York Hall of Science, with oversight by an external advisory committee of research and development professionals. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Museum of Science, Boston (MOS) and Boston University (BU) will conduct a Pilot and Feasibility Study project that leverages the current Living Laboratory (LL) model and expand it to engage high school students (teens) in experimental psychology research, science communication and science education activities. In LL, which is now an extensive network of museums and university researchers across the country, scientists and museum staff collaborate to engage children in studies on the museum floor and educate caregivers about the research. Multi-site implementation and evaluation of LL has also documented positive impacts for undergraduate researchers. Many sites are eager to extend these benefits to high school students by engaging them as practitioners within the model and by providing them with opportunities to engage in current research, education and communication, thereby helping to foster stronger youth identities with science and its applications in society. This project expands a ten-year LL partnership between MOS and BU to: 1) pilot a program in which high school students both conduct scientific research and engage the public in learning about science; 2) explore strategies for museums and universities to collaboratively engage, support and mentor high school students in science research, communication and education activities; 3) document curricular, other programmatic, and evaluation materials; and 4) convene professional participants to provide feedback on pilot materials, and assess the viability of implementing similar programs at additional sites. Guided by developmental evaluation, these activities will generate knowledge for the field, and act to increase professional capacity to integrate experiences for teens at multiple LL sites in future projects. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
We propose a thoughtful process for scientists to develop their “impact identity”, a concept that integrates scholarship in a scientific discipline with societal needs, personal preferences, capacities and skills, and one’s institutional context. Approaching broader impacts from a place of integrated identity can support cascading impacts that develop over the course of a career. We argue identity is a productive driver that can improve outcomes for scientists and for society. Widespread adoption of the concept of impact identity may also have implications for the recruitment and retention of
Supported by the National Science Foundation, the Global Soundscapes! Big Data, Big Screens, Open Ears project employs a variety of informal learning experiences to present the physics of sound and the new science of soundscape ecology. The interdisciplinary science of soundscape ecology analyzes sounds over time in different ecosystems around the world. The major components of the Global Soundscapes project are an educator-led interactive giant-screen theater show, group activities, and websites. All components are designed with both sighted and visually impaired students in mind. Multimedia
The Space and Earth Informal STEM Education (SEISE) project, led by the Arizona State University with partners Science Museum of Minnesota, Museum of Science, Boston, and the University of California Berkeley’s Lawrence Hall of Science and Space Sciences Laboratory, is raising the capacity of museums and informal science educators to engage the public in Heliophysics, Earth Science, Planetary Science, and Astrophysics, and their social dimensions through the National Informal STEM Education Network (NISE Net). SEISE will also partner on a network-to-network basis with other existing coalitions and professional associations dedicated to informal and lifelong STEM learning, including the Afterschool Alliance, National Girls Collaborative Project, NASA Museum Alliance, STAR_Net, and members of the Association of Children’s Museums and Association of Science-Technology Centers. The goals for this project include engaging multiple and diverse public audiences in STEM, improving the knowledge and skills of informal educators, and encouraging local partnerships.
In collaboration with the NASA Science Mission Directorate (SMD), SEISE is leveraging NASA subject matter experts (SMEs), SMD assets and data, and existing educational products and online portals to create compelling learning experiences that will be widely use to share the story, science, and adventure of NASA’s scientific explorations of planet Earth, our solar system, and the universe beyond. Collaborative goals include enabling STEM education, improving U.S. scientific literacy, advancing national educational goals, and leveraging science activities through partnerships. Efforts will focus on providing opportunities for learners explore and build skills in the core science and engineering content, skills, and processes related to Earth and space sciences. SEISE is creating hands-on activity toolkits (250-350 toolkits per year over four years), small footprint exhibitions (50 identical copies), and professional development opportunities (including online workshops).
Evaluation for the project will include front-end and formative data to inform the development of products and help with project decision gates, as well as summative data that will allow stakeholders to understand the project’s reach and outcomes.
This project is a Design and Development Launch Pilot (DDLP) of the NSF INCLUDES program. The goal of the project is to enhance the knowledge and applicability of science, technology, engineering, and mathematics (STEM) for a broad cross-section of people living in the U.S,-Affiliated Pacific Islands. The focus will be on water resources, which is an extremely important topic for this region and equally relevant nationally. The project will engage local community groups and schools in water monitoring, sampling, and analysis, in order to promote the benefits of science education and careers among a population that is underrepresented in these areas. Moreover, the project will improve the capabilities of the island residents for making decisions about sustainable use and protection of these scarce resources. A functioning network will be established among the islands that will have a positive impact on the health and well-being of the residents.
This project will use water as a highly relevant topic in order to involve a wide range of individuals in both general STEM learning and the basic scientific principles as applied to water resources. Specific aspects include engaging K-12, higher education, informal educators and community members to manage water resources in a sustainable fashion that will reduce disaster risk. In addition, the project will empower local communities through water literacy to make better informed, evidence-based decisions that balance the needs of diverse stakeholder groups. The overarching goal is to further advance the inclusion of underrepresented learners in STEM fields. Benefits to society will accrue by: increasing STEM learning opportunities for ~6,500 students from underserved and underrepresented Indigenous Pacific Islanders that will enhance their eligibility for STEM careers; building community resiliency through a collective impact network to resolve emerging water crises; and fostering collaboration among different constituencies in remote communities to make better-informed decisions that reflect the needs and constraints of diverse interests.
Educational approaches that provide meaningful, relevant opportunities for place-based learning have been shown to be effective models for engaging indigenous students in science. The Laulima A ‘Ike Pono (LAIP) collaboration was developed to create a place-based inclusive learning environment for engaging local community members, especially Native Hawaiians and Pacific Islanders, in scientific research at a historically significant ancient Hawaiian fishpond. The LAIP internship focused on problem-solving activities that were culturally relevant to provide a holistic STEM research experience
DATE:
TEAM MEMBERS:
Judith D. Lemus
resourceprojectGames, Simulations, and Interactives
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. The proposed project broadens the utility of Public Participation in Scientific Research (PPSR) approaches, which include citizen science, to support new angles in informal learning. It also extends previous work on interactive data visualizations in museums to encompass an element of active contribution to scientific data. To achieve these goals, this project will develop and research U!Scientist (pronounced `You, Scientist!')--a novel approach to using citizen science and learning research-based technology to engage museum visitors in learning about the process of science, shaping attitudes towards science, and science identity development. Through the U!Scientist multi-touch tabletop exhibit, visitors will: (1) interact with scientific data, (2) provide interpretations of data for direct use by scientists, (3) make statements based on evidence, and (4) visualize how their data classifications contribute to globe-spanning research projects. Visitors will also get to experience the process of science, gaining efficacy and confidence through these carefully designed interactions. This project brings together Zooniverse, experts in interactive design and learning based on large data visualizations in museums, and leaders in visitor experience and learning in science museums. Over fifty thousand museum visitors are expected to interact annually with U!Scientist through this effort. This impact will be multiplied by packaging the open-source platform so that others can easily instantiate U!Scientist at their institution.
The U!Scientist exhibit development process will follow rapid iterations of design, implementation, and revision driven by evaluation of experiences with museum visitors. It will involve close collaboration between specialists in computer science, human-computer interaction and educational design, informal science learning experts, and museum practitioners. The summative evaluation will be based on shadowing observations, U!Scientist and Zooniverse.org logfiles (i.e., automated collection of user behavior metrics), and surveys. Three key questions will be addressed through this effort: Q1) Will visitors participate in PPSR activities (via the U!Scientist touch table exhibit) on the museum floor, despite all the distractions and other learning opportunities competing for their attention? If so, who engages, for how long, and in what group configurations? Q2) If visitors do participate, will they re-engage with the content after the museum visit (i.e., continue on to Zooniverse.org)? Q3) Does engaging in PPSR via the touch table exhibit--with or without continued engagement in Zooniverse.org after the museum visit--lead to learning gains, improved understanding of the nature of science, improved attitudes towards science, and/or science identity development?
As the world is increasingly dependent upon computing and computational processes associated with data analysis, it is essential to gain a better understanding of the visualization technologies that are used to make meaning of massive scientific data. It is also essential that the infrastructure, the very means by which technologies are developed for improving the public's engagement in science itself, be better understood. Thus, this AISL Innovations in Development project will address the critical need for the public to learn how to interpret and understand highly complex and visualized scientific data. The project will design, develop and study a new technology platform, xMacroscope, as a learning tool that will allow visitors at the Science Museum of Minnesota and the Center of Science and Industry, to create, view, understand, and interact with different data sets using diverse visualization types. The xMacroscope will support rapid research prototyping of public experiences at selected exhibits, such as collecting data on a runner's speed and height and the visualized representation of such data. The xMacroscope will provide research opportunities for exhibit designers, education researchers, and learning scientists to study diverse audiences at science centers in order to understand how learning about data through the xMacroscope tool may inform definitions of data literacy. The research will advance the state of the art in visualization technology, which will have broad implications for teaching and learning of scientific data in both informal and formal learning environments. The project will lead to better understanding by science centers on how to present data to the public more effectively through visualizations that are based upon massive amounts of data. Technology results and research findings will be disseminated broadly through professional publications and presentations at science, education, and technology conferences. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project is driven by the assumption that in the digital information age, being able to create and interpret data visualizations is an important literacy for the public. The research will seek to define, measure, and advance data visualization literacy. The project will engage the public in using the xMacrocope at the Science Museum of Minnesota and at the Center of Science and Industry's (COSI) science museum and research center in Columbus, Ohio. In both museum settings the public will interact with different datasets and diverse types of visualizations. Using the xMacroscope platform, personal attributes and capabilities will be measured and personalized data visualizations will be constructed. Existing theories of learning (constructivist and constructionist) will be extended to capture the learning and use of data visualization literacy. In addition, the project team will conduct a meta-review related to different types of literacy and will produce a definition with performance measures to assess data visualization literacy - currently broadly defined in the project as the ability to read, understand, and create data visualizations. The research has potential for significant impact in the field of science and technology education and education research on visual learning. It will further our understanding of the nature of data visualization literacy learning and define opportunities for visualizing data in ways that are both personally and culturally meaningful. The project expects to advance the understanding of the role of personalization in the learning process using iterative design-based research methodologies to advance both theory and practice in informal learning settings. An iterative design process will be applied for addressing the research questions by correlating visualizations to individual actions and contributions, exploring meaning-making studies of visualization construction, and testing the xMacroscope under various conditions of crowdedness and busyness in a museum context. The evaluation plan is based upon a logic model and the evaluation will iteratively inform the direction, process, and productivity of the project.
This ChangeMakers project builds on a 2016 National Academies report finding that scientific literacy can be understood at a community level as opposed to a traditional focus on the individual. This is important since scientific knowledge is often seen as abstract and distant from the daily concerns of average citizens. A community focus shifts the spotlight away from individual learning to collective learning facilitated by trusted cultural institutions serving as social assets. This work brings together scientific expertise and community organizations to advance operational science literacy--scientific ways of problem-solving--for community leaders and functional science literacy--information and skills people can use in their daily lives--among their service populations. This will be done by gathering and sharing knowledge and developing skills and abilities to contribute to the community's overall well-being.
The New England Aquarium (NeAq) and Aquarium of the Pacific (AoP) will apply a community engagement model involving active listening, documentation, alignment of concerns and goals, and co-development of shared solutions that serves the needs of all participants. As part of the Advancing Community Science Literacy (ACSL) project, multi-disciplinary teams from NeAq, AoP and their regional partners will participate in training on the model. They will apply that training to build and implement action plans to advance community-driven responses to local environmental issues. Teams will be assessed with respect to how they use tools from their shared training, along with peer support and coaching, to make progress in engaging diverse community stakeholders. Results of the evaluation will offer insights and recommendations for informal science learning centers to serve their communities more effectively as engagement facilitators and change agents to support science literacy development and action. By applying techniques developed for cultural institutions to communicate about climate science, and combining those with techniques developed for libraries and other organizations to help meet emergent community concerns, such as storm surges and coastal flooding, it is possible to redefine the role informal science learning centers can play as part of a community culture.
ACSL is funded by the Advancing Informal STEM Learning (AISL) program which supports projects that provide multiple pathways for broadening access to and engagement in STEM learning experiences, advances innovative research on and assessment of STEM learning in informal environments, and develops understandings of deeper learning by participants.
The widespread accessibility of live streaming video now makes it possible for viewers around the world to watch live events together, including unprecedented, 24/7 views of wildlife. In addition, online technologies such as live chatting and forums have opened new possibilities for people to collaborate from locations around the world. The innovation that the projects provide is bringing these opportunities together, enabling real-time research and discussion as participants observe and annotate live streaming footage; sharing questions and insights through live Q&A sessions; and explore data with interactive visualization tools. Scientists will support the community's research interests, in contrast with traditional models of citizen science in which communities support the work of scientists. This project will enable people from diverse backgrounds and perspectives to co-create scientific investigations, including participants who might not otherwise have access to nature. The evaluation research for this project will advance the understanding of practices that enable interconnected communities of people to participate in more phases of scientific discovery, and how participation affects their learning outcomes. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of science, technology, engineering, and mathematics (STEM) learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. As such, this project will advance a new genre of Public Participation in STEM Research (PPSR). It will also advance scientific exploration using live wildlife cams and establish a database for long-term research to understand how bird behavior and reproductive success are affected by environmental change. This project aims to deepen public involvement in science, building on knowledge and relevance for STEM learning by creating an online learning environment that expands on traditional crowdsourcing models of PPSR in which participants collect data to answer questions driven by scientists. In this project, participants are involved in co-created research investigations, including asking questions, deciding what data are needed, generating data, looking for patterns, making interpretations, reviewing results, and sharing findings. The goals are to 1) create a system that involves the public more deeply in scientific research; 2) develop participants' science skills and interests; 3) increase participants' understanding of birds and the environment; 4) generate new scientific knowledge about wildlife; and 5) advance the understanding of effective project design for co-created PPSR projects at a national scale. Through iterative design and evaluation, the project will advance the understanding of the conditions that foster online collaboration and establish design principles for supporting science and discovery in online learning environments. Through scaling and quasi-experimental studies, the evaluation research will advance the understanding of how learning outcomes may be similar or different for participants engaging in different ways, whether they observe the cams and read about the investigation, process data as contributors, provide some input as collaborators, or join in most or all of the scientific process as co-creators. Despite the popularity of live wildlife cams, with millions of people watching hundreds of cams around the world, little research has been conducted on the use of live cams for collaborative work in formal or informal science education. The infrastructure and open-source framework created for this project will expand the capacity for online communities of people from diverse career backgrounds and perspectives to collaborative on solving personally meaningful questions and contribute to new knowledge. Using this project as a prototype, cam operators from around the world could build networks of cams, enabling future studies with broader scope for comparative biological studies and discoveries. Additionally, it will serve as a model for use in classrooms or for online communities exploring other scientific fields using live-streaming content in collaborative research. By involving scientists and participants from across society as collaborators and co-creators, this project can help increase public engagement with science, technology, and environmental stewardship while advancing the understanding of the natural world and informing public decision-making.