This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
This document presents the final evaluation report for the NSF-funded AISL project: "Multimodal Visitor Analytics: Investigating Naturalistic Engagement with Interactive Tabletop Science Exhibits."
In 2019, the Advancing Informal STEM Learning program at the National Science Foundation funded the Advancing Ocean Science Literacy through Immersive Virtual Reality project, a pilot/feasibility and collaborative research project between The Hydrous and the Virtual Human Interaction Lab (VHIL) at Stanford University designed to investigate how immersive virtual reality using head mounted displays can enhance ocean literacy and generate empathy towards environmental issues, particularly among high school girls from different socio-economic backgrounds. The Hydrous was responsible for designing
The Lineage project was a collaboration between Twin Cities Public Television and the Smithsonian’s National Museum of Natural History. The project included creation of a feature-length video program, a Virtual Reality game, and a set of hands-on activities designed for use by multigenerational audiences—all of which were incorporated as part of a series of seven Fossil Festival events at museums and other sites around the United States. This report presents findings from a set of external evaluation studies that examined impacts on families who participated in Fossil Festival events as well
This poster was presented at the 2021 NSF AISL Awardee Meeting.
Collaborative robots – cobots – are designed to work with humans, not replace them. What learning affordances are created in educational games when learners program robots to assist them in a game instead of being the game? What game designs work best?
This NOVA multiplatform media initiative consisted of a 2-hour nationally broadcast PBS documentary, Polar Extremes; a 10-part original digital series, Antarctic Extremes; an interactive game, Polar Lab; accompanying polar-themed digital shorts, radio stories, text reporting, and social media content; a collection of educational resources on PBS LearningMedia; and community screening events and virtual field trips for science classrooms. Across multiple media platforms the project’s video content had nearly 13 million views.
The research explored the potential for informal STEM learning
This Smart and Connected Community (SCC) project will partner with two rural communities to develop STEMports, an innovative Science, Technology, Engineering and Mathematics (STEM) learning game for workforce development. The game's activities will take players on localized Augmented Reality (AR) missions to both engage in STEM learning challenges and discover emerging STEM careers in their community, specifically highlighting innovations in the fields of sustainable agriculture and aquaculture, forest products, and renewable energy. Community Advisory Teams (CATs) and co-design teams, including youth, representatives from the targeted emerging STEM economies, and decision-makers will partner with project staff to co-design STEMports that reflect the interests, cultural contexts, and envisioned STEM industries of the future for each community.
The project will: (a) design and pilot an AR game for community STEM workforce development; (b) develop and adapt a community engagement process that optimizes community networking for co-designing the gaming application and online community; and (c) advance a scalable process for wider applications of STEMports. This project is a collaboration between the Maine Mathematics and Science Alliance and the Field Day Lab at the University of Wisconsin-Madison to both build and research the co-designing of a SCC based within an AR environment. The project will contribute knowledge to the informal STEM learning, community development, and education technology fields in four major ways:
Deepening the understanding of how innovative technological tools support rural community STEM knowledge building as well as STEM identity and workforce interest.
Identifying design principles for co-designing the STEMports community related to the technological design process.
Developing social network approaches and analytics to better understand the social dimensions and community connections fostered by the STEMport community.
Understanding how participants' online and offline interactions with individuals and experiences builds networks and knowledge within a SCC.
With the scaling of use by an ever-growing community of players, STEMports will provide a new AR-based genre of public participation in STEM and collective decision making. The research findings will add to the emerging literature on community-wide education, innovative education technologies, informal STEM learning (especially place-based learning and STEM ecosystems), and participatory design research.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Sense-making with data through the process of visualization—recognizing and constructing meaning with these data—has been of interest to learning researchers for many years. Results of a variety of data visualization projects in museums and science centers suggest that visitors have a rudimentary understanding of and ability to interpret the data that appear in even simple data visualizations. This project supports the need for data visualization experiences to be appealing, accommodate short and long-term exploration, and address a range of visitors’ prior knowledge. Front-end evaluation
The project will develop and research how an emerging technology, immersive virtual reality (IVR) using head mounted displays (HMDs), can enhance ocean literacy and generate empathy towards environmental issues. Recent advances in design have resulted in HMDs that provide viscerally realistic and immersive experiences that situate participants in underwater or other remote environments. IVR can provide many people with virtual access to these environments, including persons with disabilities, people living away from coastal areas, or those who lack access for other reasons (e.g., low-income families, underserved/underrepresented communities, persons untrained in diving). The project will develop a high quality 360-degree underwater film that includes live action footage, animation, and interactive elements. The IVR experience will take the participant through an immersive underwater journey of a Pacific reef, using realistic visualizations, narrative, and a compelling story to engage participants in learning the ecology and biology of coral reefs, as well as the impacts of climate change and human disturbances on ocean ecosystems. In addition to the IVR ocean journey, the project will integrate interactive functionality of being on a reef during mass coral spawning, an annual natural phenomenon through which coral reefs replenish their populations. With hand-held controllers, participants will be able to "swim" through the water, watch the degraded reef recover and grow and will have the ability to change the rate of coral recovery and learn how increases in temperature impede coral recovery. While research has been conducted on early, desk-top versions of IVR, the potential impact of IVR on learning is still unclear. The research findings will help guide the development of IVR for use in informal STEM environments such as aquariums, zoos, science museums, and others. The IVR experience will be shared on online platforms for home viewing, at film festivals and conferences, and in informal learning environments.
The project addresses the need for research on the impacts of IVR devices as it become more affordable and more widely used at home and in other informal and formal environments. Few studies have investigated how design elements impact the user in IVR, in which the increased immersion affects the stimuli perception and cognitive processing. The research will assess the learning affordances and impacts of the IVR experience on participant ocean literacy (adapting items from an existing ocean literacy survey), environmental empathy/feelings of presence (naturalistic observations and post-experience interviews), and perceived self-efficacy (pre-post survey, post-interview interviews). In addition, the project will research how segmentation (i.e., a continuous experience vs. an experience with breaks), generative learning tasks (hands-on experiences and interactive during IVR), and gender of the narrator in an IVR experience supports learning about ocean environments. Researchers will collect data from students attending high schools with predominantly minority student enrollments. Research findings will be widely shared through peer-reviewed publications, conference presentations, and publications for educators and designers.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Robots and robotics excite and challenge youths and adults. Unfortunately, the cost of purchasing robots or building useful robots is prohibitive for many low resource individuals and groups. This project will relieve this expense and provide an opportunity for resource limited individuals to experience the thrilling aspects of robotics by building a computer game that simulates robotic action. This project uses co-robotics wherein the participating player programs an avatar to assist in a symbiotic manner to achieve the goals of the game and participant. The game will provide access to the ideas and concepts such as programing, computational thinking and role assumption. The overarching goals are (1) to engage low-resource learners in STEM education through robotics in out-of-school spaces, and (2) to update the field of robotics-base STEM education to integrate the co-robotics paradigm.
This project is designed to gain knowledge on how co-robotics can be used in the informal education sector to facilitate the integration of computational science with STEM topics and to expand the educational use of co-robotics. Because the concept of co-robotics is new, a designed-based research approach will be used to build theoretical knowledge and knowledge of effective interventions for helping participants learn programing and computational thinking. Data will be collected from several sources including surveys, self-reports, in game surveys, pre and post-tests. These data collection efforts will address the following areas: Technology reliability, Resolution of cognitive tension around co-play, Accelerate discovery and initial engagement, Foster role-taking and interdependence with co-robots, Investigate social learning, and Validate measures using item response theory analysis. The DBR study questions are:
1.What design principles support the development of P3Gs that can effectively attract initial engagement in a free-choice OST space that offers large numbers of competing options? 2.What design principles support a P3G gameplay loop that enables learning of complex skills, computational thinking and co-robotics norms, and building of individual and career interest over the course of repeated engagement?
3.What design principles support P3Gs in attaining a high rate of re-engagement within low-resource OST settings? 4.What kinds of positive impact can P3Gs have on their proximal and distal environment? In addition, the project will research these questions about design: 1.What technical and game design features are needed to accommodate technological interruption? 2.What design elements or principles mitigate competition for cognitive resources between real-time play and understanding the co-robotic's behavior in relation to the code the player wrote for it? 3.What design elements are effective at getting learners in OST settings to notice and start playing the game? 4.What designs are effective at encouraging learners to engage with challenging content, particularly the transition from manual play to co-play? 5.What design elements help players develop a stake in the role the game offers? 6.What social behaviors emerge organically around a P3G prototype that is not designed to evoke specific social interactions?
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.