In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
In 2019, the Advancing Informal STEM Learning program at the National Science Foundation funded the Advancing Ocean Science Literacy through Immersive Virtual Reality project, a pilot/feasibility and collaborative research project between The Hydrous and the Virtual Human Interaction Lab (VHIL) at Stanford University designed to investigate how immersive virtual reality using head mounted displays can enhance ocean literacy and generate empathy towards environmental issues, particularly among high school girls from different socio-economic backgrounds. The Hydrous was responsible for designing
This poster was presented at the 2021 NSF AISL Awardee Meeting.
Collaborative robots – cobots – are designed to work with humans, not replace them. What learning affordances are created in educational games when learners program robots to assist them in a game instead of being the game? What game designs work best?
This NOVA multiplatform media initiative consisted of a 2-hour nationally broadcast PBS documentary, Polar Extremes; a 10-part original digital series, Antarctic Extremes; an interactive game, Polar Lab; accompanying polar-themed digital shorts, radio stories, text reporting, and social media content; a collection of educational resources on PBS LearningMedia; and community screening events and virtual field trips for science classrooms. Across multiple media platforms the project’s video content had nearly 13 million views.
The research explored the potential for informal STEM learning
This project tackles the urgent needs of the nation to engage people of all ages in computational thinking and help them learn basic computer science concepts with a unique and innovative approach of structured in-game computer program coding. Researchers will explore the design and development of a 3D puzzle-based game, called May's Journey, in which players solve an environmental maze by using the game's pseudo code to manipulate game objects. The game is designed to teach introductory but foundational concepts of computer programming including abstraction, modularity, reusability, and debugging by focusing players on logic and concepts while asking them to type simple instructions in a simplified programming language designed for novices. The game design in this project differs from today's block-based programming learning approaches that are often too far from actual computer code, and also differs from professional programming languages which are too complex for novices. The game and its embedded programming language learning are designed to be responsive to the progress of the learner throughout the game, transitioning from pseudo code to the embedded programming language itself. Error messages for debugging are also designed to be adaptive to players' behavior in the game. Using extensive log data collected from people playing the game, researchers can study how people learn computer programming. Such knowledge can advance understanding of the learning processes in computer programming education. Additionally, this work emphasizes the use of games as informal learning environments as they are accessible and fun, drawing attention and retention of many learners of different age groups with the potential to change attitudes towards computer programming across different populations. This project is co-funded by the STEM + Computing (STEM+C) program that supports research and development to understand the integration of computing and computational thinking in STEM learning, and the Advancing Informal STEM Learning (AISL) program that funds innovative research, approaches and resources for use in a variety of settings with its overall strategy to enhance learning in informal environments.
The project's formative and summative evaluation methods, including surveys, expert reviews of learners' computer code developed in the game, and interviews, are used to gauge learners' engagement as well as learning. In exploring learning, researchers aim to understand how players build implicit computer science knowledge through gameplay and how that gameplay relates to their performance on external transfer tasks. The project will answer the following three research questions: (1) Can observers reliably detect and label patterns of gameplay that provide evidence of learning or misconceptions regarding the four computer science constructs - abstraction, modularity, debugging and semantics - that learners exhibit playing May's Journey? (2) How does learner's implicit knowledge of these computer science constructs change over time and do those patterns vary by gender and prior programming experiences? (3) Is there a strong correlation between implicit learning measures and transfer of CS concepts: modularity, debugging, semantics, and abstraction? How do these correlations vary across elements of the game? This work will result in several outcomes: game design metaphors tested for their learning and engagement value that can be abstracted and embedded in different games. This project will also contribute patterns and an understanding of how people learn and engage in problem solving using concepts of abstraction, modularity, debugging and semantics. These outcomes will lead to advancement in knowledge in the learning sciences as well as the design of educational games that enrich STEM learning, particularly in programming and computational thinking. In addition, this project will engage female participants and underserved populations through partnering organizations including National Girls Collaborative project.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Magy Seif El-Nasr
resourceprojectGames, Simulations, and Interactives
EMERGE in STEM (Education for Minorities to Effectively Raise Graduation and Employment in STEM) is a NSF INCLUDES Design and Development Launch Pilot. This project addresses the broadening participation challenge of increasing participation of women, the at-risk minority population, and the deaf in the STEM workforce. The project incorporates in and out-of-school career awareness activities for grades 4-12 in a high poverty community in Guilford County, North Carolina. EMERGE in STEM brings together a constellation of existing community partners from all three sectors (public, private, government) to leverage and expand mutually reinforcing STEM career awareness and workforce development activities in new ways by using a collective impact approach.
This project builds on a local network to infuse career exposure elements into the existing mutually reinforcing STEM activities and interventions in the community. A STEM education and career exposure software, Learning Blade, will be used to reach approximately 15,000 students. A shared measurement system and assessment process will contribute to the evaluation of the effectiveness of the collective impact strategies, the implementation of mutually reinforcing activities across the partnership and the extent to which project efforts attract students to consider STEM careers.
DATE:
-
TEAM MEMBERS:
Gregory MontyMargaret KanipesMalcolm SchugSteven Jiang
Women continue to be underrepresented in computer science professions. In 2015, while 57% of professional occupations in the U.S. were held by women, only 25% of computing occupations were held by women. Furthermore, the share of computer science degrees going to women is smaller than any STEM field, even though technology careers are the most promising in terms of salaries and future growth. Research suggests that issues contributing to this lack of computer science participation begin early and involve complex social and environmental factors, including girls' perception that they do not belong in computer science classes or careers. Computer science instruction often alienates girls with irrelevant curriculum; non-collaborative pedagogies; a lack of opportunities to take risks or make mistakes; and a heavy reliance on lecture instead of hands-on, project-based learning. Computer science experiences that employ research-based gender equitable best practices, particularly role modeling, can help diminish the gender gap in participation. In response to this challenge, Twin Cities PBS (TPT), the National Girls Collaborative (NGC) and Code.org will lead Code: SciGirls! Media for Engaging Girls in Computing Pathways, a three-year project designed to engage 8-13 year-old girls in coding through transmedia programming which inspires and prepares them for future computer science studies and career paths. The project includes five new PBS SciGirls episodes featuring girls and female coding professionals using coding to solve real problems; a new interactive PBSKids.org game that allows children to develop coding skills; nationwide outreach programming, including professional development for informal educators and female coding professionals to facilitate activities for girls and families in diverse STEM learning environments; a research study that will advance understanding of how the transmedia components build girls' motivation to pursue additional coding experiences; and a third-party summative evaluation.
Code: SciGirls! will foster greater awareness of and engagement in computer science studies and career paths for girls. The PBS SciGirls episodes will feature girls and female computer science professionals using coding to solve real-world challenges. The project's transmedia component will leverage the television content into the online space in which much of 21st century learning takes place. The new interactive PBSKids.org game will use a narrative framework to help children develop coding skills. Drawing on narrative transportation theory and character identification theory, TPT will commission two exploratory knowledge-building studies to investigate: To what extent and how do the narrative formats of the Code: SciGirls! online media affect girls' interest, beliefs, and behavioral intent towards coding and code-related careers? The studies aim to advance understanding of how media builds girls' motivation to pursue computer science experiences, a skill set critical to building tomorrow's workforce. The project team will also raise educators' awareness about the importance of gender equitable computer science instruction, and empower them with best practices to welcome, prepare and retain girls in coding. The Code: SciGirls! Activity Guide will provide educators with a relevant resource for engaging aspiring computer scientists. The new media and guide will also reside on PBSLearningMedia.org, reaching 1.2 million teachers, and will be shared with thousands of educators across the SciGirls CONNECT and National Girls Collaborative networks. The new episodes are anticipated to reach 92% of U.S. TV households via PBS, and the game at PBSKids.org will introduce millions of children to coding. The summative evaluation will examine the reach and impact of the episodes, game and new activities. PIs will share research findings and project resources at national conferences and will submit to relevant publications. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
Computer Science (CS) education at the middle school level using educational games has seen recent growth and shown promising results. Typically these games teach the craft of programming and not the perspectives required for computational thinking, such as abstraction and algorithm design, characteristic of a CS curriculum. This research presents a game designed to teach computational thinking via the problem of minimum spanning trees to middle school students, a set of evaluation instruments, and the results of an experimental pilot study. Results show a moderate increase in minimum spanning
In support of a summative evaluation of SciGirls Season Three, Multimedia Research developed a scale to assess preteen girls’ interest in nature and science. The work was sponsored by Twin Cities PBS under NSF Grant No. 1323713. Multimedia Research developed, piloted, validated and implemented the GINSS: A nine statement Likert scale constructed to reveal girls’ strength of interest in nature and science activities. Researchers and evaluators are encouraged to use this scale to extend its application. Please email if you eventually use the scale in your research or evaluation: Flagg
Brigham Young University and the University of Maryland, in partnership with the Smithsonian Institution, the Computer History Museum, and NASA, plus leading game designers, educators, scientists, and researchers, will conduct research on the design and development of two large-scale Alternate Reality Games (ARGs) based on deep-time science in astrobiology, astrophysics, and interplanetary space travel. The project will iteratively design and test two distinct types of ARGs (closed- and open-ended) to study the effects of these ARGs on STEM learning. The ARGs will be based upon the Next Generation Science Standards (NGSS), affording learners with intensive, self-driven, and scaffolded scientific learning and will be aimed at attracting girls and other groups historically underrepresented in science and technology. Each ARG will be designed by NASA scientists, educators and education researchers, and game-based learning experts and will be highly interactive: engaging learners in collaborative investigations in real and virtual worlds to collect scientific data, conduct data analysis, and contribute scientific evidence that will help solve scientific questions within a science-based narrative derived from real world problems that will develop learners' computational thinking skills in a collaborative, participatory virtual learning environment. Combining data from web and social media analytics, player interviews, surveys, and user-generated content, researchers, and evaluation experts at UXR who will provide an outcomes-based evaluation, including front-end, formative, remedial, and summative evaluations, will establish the properties of ARGs that most effectively advance informal STEM learning outcomes. By comparing open-ended and closed-ended ARGs, the PIs will be able to assess the relative strengths and weaknesses of two distinct approaches to Alternate Reality Game design. The project team will test the hypothesis that open-ended, user-generated content will support inquiry-based learning, peer-to-peer learning, and life-wide and life-deep learning, while close-ended, narrative-rich ARGs will support specific transfer of STEM knowledge, collaboration, and problem solving. To help ensure that the games appeal to their target audiences, the project team will adopt co-design methods, enlisting the creative input of participating teens at each stage of the design process. Supplementary materials and lesson plans developed in close consultation with teachers, librarians, teens, and external stakeholders will enable the ARGs to be widely and effectively used as a model in museums, classrooms, libraries, and after-school programs. The proposed ARGs represent a unique environment to test learning principles that enable players to bridge their learning through transmedia across multiple contexts and test the effects of collaboration with massive numbers of concurrent players. As a result, the project should yield insights on how learning principles can be adopted and re-appropriated for emerging learning environments, including those that that might be crowd-sourced. The research is well grounded in the literature and the PIs do an excellent job of mapping ARG design principles to the pertinent learning science research, providing a clear sense of the particular affordances of the genre that should lead to new understandings. The approach has profound implications for the way we might teach the next generation of students. The ability to mix problem solving and learning in virtual spaces with experiences and data derived from the physical world could dramatically change how we understand the role of technology in education.
Produced by Twin Cities Public Television, St. Paul, MN, and sponsored by the National Science Foundation, SciGirls (SG) is a multimedia project for upper grade-school and middle-school tweens. Weekly half-hour episodes are tied in with web and outreach activities in the fields of science, technology and engineering. Multimedia Research, an independent evaluation group implemented a summative evaluation of the SG Season Two multimedia project. Fifth grade girls (N = 87) viewed three shows over three weeks. They could visit the SG website at any time but were required to visit and play a Pick'm
DATE:
TEAM MEMBERS:
Barbara FlaggTwin Cities Public Television
This planning grant seeks funding for demonstrating proof of concept for the proposed traveling exhibition "Inside Phoebe's Field." It is an outgrowth of the prior NSF project (9979287) that resulted in an interactive book for middle-school age girls. This project extends the book through exploring the feasibility of using mixed reality in an exhibit format as well as virtual environments using the Internet. Intellectual Merit: The project will further the technology-enhanced three-dimensional storytelling approach by development of selected science exhibit prototypes. The project team combines academic, museum and professional expertise, as well as a partnership with the Girl Scouts of the USA. Broader Impact: This project will provide the ISE field valuable feedback about the use of metaphor in storytelling in a science-based exhibition, as well as about the application of new technologies (virtual reality, augmented reality technologies and interactive electromechanical events) for creating exhibit experiences. At the same time it will further the development of a national traveling science exhibition whose primary target audience will be girls from ages 10 to 13.
DATE:
-
TEAM MEMBERS:
Mitzi VernonKatherine CennamoIrene GoodmanMichael ErmannSteve HarrisonMargarita McGrath