In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
This report is the summative evaluation of Moon Adventure Game. The Moon Adventure Game is a challenge-based immersive game, inspired by “escape room” experiences, which asks visitors to take on activities to help them think about what people might need to live and work on the Moon.
This project includes the development of a toolkit of new hands-on facilitated museum activities, and a mobile app with both app-based activities and do-it-yourself (DIY) activities. This evaluation report focuses on the formative evaluation of three app activities that are being added to the DIY app series.
This assessment serves as the summative assessment of the IMLS-funded project at KU Biodiversity Institute and Natural History Museum: Natural History Mystery: Immersing families in a problem-solving game using museum collections. The assessment employs a mixed methods approach, in which both quantitative and qualitative data are collected. More specifically, quantitative data are generated from surveys that are administered to participants at the beginning and end of the game and analyzed by using descriptive statistics (i.e., mean, standard deviation, and histogram) and paired sample t-test
This poster was presented at the 2021 NSF AISL Awardee Meeting.
Collaborative robots – cobots – are designed to work with humans, not replace them. What learning affordances are created in educational games when learners program robots to assist them in a game instead of being the game? What game designs work best?
Robots and robotics excite and challenge youths and adults. Unfortunately, the cost of purchasing robots or building useful robots is prohibitive for many low resource individuals and groups. This project will relieve this expense and provide an opportunity for resource limited individuals to experience the thrilling aspects of robotics by building a computer game that simulates robotic action. This project uses co-robotics wherein the participating player programs an avatar to assist in a symbiotic manner to achieve the goals of the game and participant. The game will provide access to the ideas and concepts such as programing, computational thinking and role assumption. The overarching goals are (1) to engage low-resource learners in STEM education through robotics in out-of-school spaces, and (2) to update the field of robotics-base STEM education to integrate the co-robotics paradigm.
This project is designed to gain knowledge on how co-robotics can be used in the informal education sector to facilitate the integration of computational science with STEM topics and to expand the educational use of co-robotics. Because the concept of co-robotics is new, a designed-based research approach will be used to build theoretical knowledge and knowledge of effective interventions for helping participants learn programing and computational thinking. Data will be collected from several sources including surveys, self-reports, in game surveys, pre and post-tests. These data collection efforts will address the following areas: Technology reliability, Resolution of cognitive tension around co-play, Accelerate discovery and initial engagement, Foster role-taking and interdependence with co-robots, Investigate social learning, and Validate measures using item response theory analysis. The DBR study questions are:
1.What design principles support the development of P3Gs that can effectively attract initial engagement in a free-choice OST space that offers large numbers of competing options? 2.What design principles support a P3G gameplay loop that enables learning of complex skills, computational thinking and co-robotics norms, and building of individual and career interest over the course of repeated engagement?
3.What design principles support P3Gs in attaining a high rate of re-engagement within low-resource OST settings? 4.What kinds of positive impact can P3Gs have on their proximal and distal environment? In addition, the project will research these questions about design: 1.What technical and game design features are needed to accommodate technological interruption? 2.What design elements or principles mitigate competition for cognitive resources between real-time play and understanding the co-robotic's behavior in relation to the code the player wrote for it? 3.What design elements are effective at getting learners in OST settings to notice and start playing the game? 4.What designs are effective at encouraging learners to engage with challenging content, particularly the transition from manual play to co-play? 5.What design elements help players develop a stake in the role the game offers? 6.What social behaviors emerge organically around a P3G prototype that is not designed to evoke specific social interactions?
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Casual games are everywhere. People play them throughout life to pass the time, to engage in social interactions, and to learn. However, their simplicity and use in distraction-heavy environments can attenuate their potential for learning. This experimental study explored the effects playing an online, casual game has on awareness of human biological systems. Two hundred and forty-two children were given pretests at a Museum and posttests at home after playing either a treatment or control game. Also, 41 children were interviewed to explore deeper meanings behind the test results. Results show
Supported by the National Science Foundation, the Global Soundscapes! Big Data, Big Screens, Open Ears project employs a variety of informal learning experiences to present the physics of sound and the new science of soundscape ecology. The interdisciplinary science of soundscape ecology analyzes sounds over time in different ecosystems around the world. The major components of the Global Soundscapes project are an educator-led interactive giant-screen theater show, group activities, and websites. All components are designed with both sighted and visually impaired students in mind. Multimedia
EMERGE in STEM (Education for Minorities to Effectively Raise Graduation and Employment in STEM) is a NSF INCLUDES Design and Development Launch Pilot. This project addresses the broadening participation challenge of increasing participation of women, the at-risk minority population, and the deaf in the STEM workforce. The project incorporates in and out-of-school career awareness activities for grades 4-12 in a high poverty community in Guilford County, North Carolina. EMERGE in STEM brings together a constellation of existing community partners from all three sectors (public, private, government) to leverage and expand mutually reinforcing STEM career awareness and workforce development activities in new ways by using a collective impact approach.
This project builds on a local network to infuse career exposure elements into the existing mutually reinforcing STEM activities and interventions in the community. A STEM education and career exposure software, Learning Blade, will be used to reach approximately 15,000 students. A shared measurement system and assessment process will contribute to the evaluation of the effectiveness of the collective impact strategies, the implementation of mutually reinforcing activities across the partnership and the extent to which project efforts attract students to consider STEM careers.
DATE:
-
TEAM MEMBERS:
Gregory MontyMargaret KanipesMalcolm SchugSteven Jiang
Women continue to be underrepresented in computer science professions. In 2015, while 57% of professional occupations in the U.S. were held by women, only 25% of computing occupations were held by women. Furthermore, the share of computer science degrees going to women is smaller than any STEM field, even though technology careers are the most promising in terms of salaries and future growth. Research suggests that issues contributing to this lack of computer science participation begin early and involve complex social and environmental factors, including girls' perception that they do not belong in computer science classes or careers. Computer science instruction often alienates girls with irrelevant curriculum; non-collaborative pedagogies; a lack of opportunities to take risks or make mistakes; and a heavy reliance on lecture instead of hands-on, project-based learning. Computer science experiences that employ research-based gender equitable best practices, particularly role modeling, can help diminish the gender gap in participation. In response to this challenge, Twin Cities PBS (TPT), the National Girls Collaborative (NGC) and Code.org will lead Code: SciGirls! Media for Engaging Girls in Computing Pathways, a three-year project designed to engage 8-13 year-old girls in coding through transmedia programming which inspires and prepares them for future computer science studies and career paths. The project includes five new PBS SciGirls episodes featuring girls and female coding professionals using coding to solve real problems; a new interactive PBSKids.org game that allows children to develop coding skills; nationwide outreach programming, including professional development for informal educators and female coding professionals to facilitate activities for girls and families in diverse STEM learning environments; a research study that will advance understanding of how the transmedia components build girls' motivation to pursue additional coding experiences; and a third-party summative evaluation.
Code: SciGirls! will foster greater awareness of and engagement in computer science studies and career paths for girls. The PBS SciGirls episodes will feature girls and female computer science professionals using coding to solve real-world challenges. The project's transmedia component will leverage the television content into the online space in which much of 21st century learning takes place. The new interactive PBSKids.org game will use a narrative framework to help children develop coding skills. Drawing on narrative transportation theory and character identification theory, TPT will commission two exploratory knowledge-building studies to investigate: To what extent and how do the narrative formats of the Code: SciGirls! online media affect girls' interest, beliefs, and behavioral intent towards coding and code-related careers? The studies aim to advance understanding of how media builds girls' motivation to pursue computer science experiences, a skill set critical to building tomorrow's workforce. The project team will also raise educators' awareness about the importance of gender equitable computer science instruction, and empower them with best practices to welcome, prepare and retain girls in coding. The Code: SciGirls! Activity Guide will provide educators with a relevant resource for engaging aspiring computer scientists. The new media and guide will also reside on PBSLearningMedia.org, reaching 1.2 million teachers, and will be shared with thousands of educators across the SciGirls CONNECT and National Girls Collaborative networks. The new episodes are anticipated to reach 92% of U.S. TV households via PBS, and the game at PBSKids.org will introduce millions of children to coding. The summative evaluation will examine the reach and impact of the episodes, game and new activities. PIs will share research findings and project resources at national conferences and will submit to relevant publications. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
Well-designed educational games represent a promising technology for increasing students interest in and learning of STEM topics such as physics. This project will research how to optimally combine and embed dynamic assessment and adaptive learning supports within an engaging game design to build effective educational games. The project will add enhancements to a physics game called Physics Playground. The general goal of this research is to test a valid methodology that can be used in the design of next-generation learning games. The enhancement of Physics Playground will leverage the popularity of video games to capture and sustain student attention and teach physics to a much broader audience than is currently the case in traditional physics classrooms. To be most effective, this new genre of learning games needs to not only be highly engaging as a game but also to provide real-time assessment and feedback to students; support understanding of science content (i.e.,Newtonian physics); be accessible to beginners; accommodate a range of proficiencies and interests; and support equity. The research will have particular relevance to designers developing other science games and simulation by providing information about the kinds of learning supports and feedback to students are most effective in promoting engagement and learning. The project is supported by the Cyberlearning and Future Learning Technologies Program, which funds efforts that will help envision the next generation of learning technologies and advance what we know about how people learn in technology-rich environments. Cyberlearning Exploration (EXP) Projects explore the viability of new kinds of learning technologies by designing and building new kinds of learning technologies and studying their possibilities for fostering learning and challenges to using them effectively.
The project will systematically develop, test, and evaluate ways to integrate engaging, dynamic learning supports in Physics Playground to teach formal conceptual physics competencies. More generally, the project aims to advance the learning sciences, particularly in the fields of adaptivity and assessment in educational technology. Using a design-based research approach spanning three years, the research team will: (1) develop and test the effectiveness of various learning support features included in the game in Year 1; (2) develop and test an adaptive algorithm to manage the progression of difficulty in game levels in Year 2; and (3) test learning supports and adaptive sequencing in a controlled evaluation study. This research will provide evidence of the instructional effectiveness of an educational game designed using principles of instructional, game, and assessment design. It will advance understanding of the contributions of different kinds of learning supports (e.g., visualizations and explanations) and adaptivity to game-based learning and contribute to the design of next-generation learning games that successfully blur the distinction between assessment and learning. The project will generate research findings that can be incorporated into other types of STEM learning games.
DATE:
-
TEAM MEMBERS:
Valerie ShuteRussell AlmondFengfeng Ke
This project, a collaboration of teams at Georgia Institute of Technology, Northwestern University, and the Museum of Design Atlanta and the Museum of Science and Industry in Chicago, will investigate how to foster engagement and broadening participation in computing by audiences in museums and other informal learning environments that can transfer to at-home and in-school engagement (and vice versa). The project seeks to address the national need to make major strides in developing computing literacy as a core 21st century STEM skill. The project will adapt and expand to new venues their current work on their EarSketch system which connects computer programming concepts to music remixing, i.e. the manipulation of musical samples, beats and effects. The initiative involves a four-year process of iteratively designing and developing a tangible programming environment based on the EarSketch learning environment. The team will develop three new applications: TuneTable, a multi-user tabletop exhibit for museums; TunePad, a smaller version for use at home and in schools; and an online connection between the earlier EarSketch program and the two new devices.
The goal is to: a) engage museum learners in collaborative, playful programming experiences that create music; b) direct museum learners to further learning and computational music experiences online with the EarSketch learning environment; c) attract EarSketch learners from local area schools to visit the museum and interact with novice TuneTable users, either as mentors in museum workshops or museum guests; and d) inform the development of a smaller scale, affordable tangible-based experience that could be used at homes or in smaller educational settings, such as classrooms and community centers. In addition to the development of new learning experiences, the project will test the hypothesis that creative, playful, and social engagement in the arts with computer programming across multiple settings (e.g. museums, homes, and classrooms) can encourage: a) deeper learner involvement in computer programming, b) social connections to other learners, c) positive attitudes towards computing, and d) the use and recognition of computational concepts for personal expression in music. The project's knowledge-building efforts include research on four major questions related to the goals and evaluation processes conducted by SageFox on the fidelity of implementation, impact, success of the exhibits, and success of bridging contexts. Methods will draw on the Active Prolonged Engagement approach (unobtrusive observation, interviews, tracking-and-timing, data summaries and team debriefs) as well as Participatory Action Research methods.
This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
DATE:
-
TEAM MEMBERS:
Michael HornBrian MagerkoJason Freeman