This project tackles the urgent needs of the nation to engage people of all ages in computational thinking and help them learn basic computer science concepts with a unique and innovative approach of structured in-game computer program coding. Researchers will explore the design and development of a 3D puzzle-based game, called May's Journey, in which players solve an environmental maze by using the game's pseudo code to manipulate game objects. The game is designed to teach introductory but foundational concepts of computer programming including abstraction, modularity, reusability, and debugging by focusing players on logic and concepts while asking them to type simple instructions in a simplified programming language designed for novices. The game design in this project differs from today's block-based programming learning approaches that are often too far from actual computer code, and also differs from professional programming languages which are too complex for novices. The game and its embedded programming language learning are designed to be responsive to the progress of the learner throughout the game, transitioning from pseudo code to the embedded programming language itself. Error messages for debugging are also designed to be adaptive to players' behavior in the game. Using extensive log data collected from people playing the game, researchers can study how people learn computer programming. Such knowledge can advance understanding of the learning processes in computer programming education. Additionally, this work emphasizes the use of games as informal learning environments as they are accessible and fun, drawing attention and retention of many learners of different age groups with the potential to change attitudes towards computer programming across different populations. This project is co-funded by the STEM + Computing (STEM+C) program that supports research and development to understand the integration of computing and computational thinking in STEM learning, and the Advancing Informal STEM Learning (AISL) program that funds innovative research, approaches and resources for use in a variety of settings with its overall strategy to enhance learning in informal environments.
The project's formative and summative evaluation methods, including surveys, expert reviews of learners' computer code developed in the game, and interviews, are used to gauge learners' engagement as well as learning. In exploring learning, researchers aim to understand how players build implicit computer science knowledge through gameplay and how that gameplay relates to their performance on external transfer tasks. The project will answer the following three research questions: (1) Can observers reliably detect and label patterns of gameplay that provide evidence of learning or misconceptions regarding the four computer science constructs - abstraction, modularity, debugging and semantics - that learners exhibit playing May's Journey? (2) How does learner's implicit knowledge of these computer science constructs change over time and do those patterns vary by gender and prior programming experiences? (3) Is there a strong correlation between implicit learning measures and transfer of CS concepts: modularity, debugging, semantics, and abstraction? How do these correlations vary across elements of the game? This work will result in several outcomes: game design metaphors tested for their learning and engagement value that can be abstracted and embedded in different games. This project will also contribute patterns and an understanding of how people learn and engage in problem solving using concepts of abstraction, modularity, debugging and semantics. These outcomes will lead to advancement in knowledge in the learning sciences as well as the design of educational games that enrich STEM learning, particularly in programming and computational thinking. In addition, this project will engage female participants and underserved populations through partnering organizations including National Girls Collaborative project.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Pilot and Feasibility study will investigate strategies for enhancing the mathematics in museum-based making and tinkering activities and lay the foundation for a full research study on broadening family participation in mathematics through making. This proposal builds directly on the NSF-funded Math in the Making convening. During this convening, questions about how to authentically highlight and enhance the mathematics in making and tinkering experiences, and how different math-enhancement approaches might influence learner experiences and outcomes, emerged as critical issues for researchers, educators, and mathematicians alike. The project aims to provide a practical lens to help researchers and educators connect topics across STEM with making and tinkering experiences. The project also seeks to advance theoretical understandings of museum-based learning by exploring ways that activity design and facilitation strategies influence how visitors understand the nature and goals of the experience and, in turn, how these visitor experiences shape learning outcomes. The project is designed to explore the most promising of these math-enhancement strategies in more depth, to propose as a next project and develop a theoretical framework for understanding and describing how these strategies influence how families understand and engage with the mathematics in maker experiences. Through several culturally-responsive approaches developed in collaboration with community-based organizations, the project will research how mathematics in maker experiences influences participant engagement and learning. The project will culminate in the design of a research study. Reports and resources developed by the project will be broadly disseminated to researchers, mathematicians, and educators. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Explore the Science of Spring: A Live Media Event is an Innovations in Development project produced by the signature PBS series Nature. The new primetime series Spring LIVE (working title) will break the frame of a traditional documentary, letting viewers themselves explore the dramatic seasonal changes of spring through the immediacy of live television. On-camera hosts, scientists and naturalists in locations across the U.S., and scores of citizen scientists will use observation and scientific inquiry to explore the workings of nature during this season of rebirth. The unfolding stories of seasonal change will illuminate larger scientific insights--into the biodiversity of species in habitats, the interconnectedness of plants and animals in diverse ecosystems, the global phenomenon of species migration, and how spring "green-up" can be affected by environmental change--while inspiring appreciation for species conservation and habitat preservation. Spring LIVE is conceived as an ongoing series, with this inaugural season composed of three one-hour programs broadcast live on three consecutive nights, along with real-time interactions via Facebook. Reaching long-standing Nature viewers (2.5 million per episode), Spring LIVE will seek to turn mature adults and diverse families into citizen science doers, and leverage younger Nature online audiences through social media and community engagement in partnership with citizen science projects.
Spring LIVE will build public knowledge of and engagement in phenology and citizen science. The project will also conduct knowledge-building research on the effectiveness of Facebook as a science learning tool. It will experiment with eliciting audience participation via Facebook within the live shows to generate synchronous, second-screen thought and discussion. An exploratory study by Multimedia Research will look at the impact of this feature, addressing the question: To what extent and how does Facebook interactivity within live science shows impact adult engagement, learning and motivation? Spring LIVE will also engage multiple partners to expand reach and impact and build capacity in their fields. National partners include the National Park Service and Next Avenue; citizen science partners include Celebrate Urban Birds, National Phenology Network, Monarch Blitz, and SciStarter, among others. PBS stations will work with these organizations to involve diverse, intergenerational audiences in observation of nature and seasonal change. Project evaluation, implemented by Knight Williams Research Communications, will focus on the impact of live television on science learning, and the success of the integration of citizen science projects on air, online, and in communities. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Exploratorium in San Francisco, in collaboration with social science researchers at Arizona State University and Rutgers University, will conduct a two-year Pilots and Feasibility Studies project that will test a new model for integrating museum exhibits with public participation in scientific research, called Exhibit-Based PPSR (E-PPSR). The team will create a mini-exhibition about social psychology that (a) engages science museum visitors in investigating and reflecting on social factors that promote cooperation or not, (b) builds public awareness of the science of social psychology and (c) generates valid data for academic research in this area.
The Exploratorium will build on its long history of creating participatory exhibits that engage visiting groups in social science learning and in self-reflective metacognition. This expands beyond the typical lab-in-the-museum in which individual visitors interact with researchers' laptops or simple props. The model takes the exhibit experiences further, injecting the dimension of public participation in social psychology research. By voluntarily and anonymously contributing demographic and response data, large numbers of people from mixed-age groups will help social scientists gain new insights into cooperation among a broader sample of humanity than are usually studied in university labs. The E-PPSR lab is always available, rather than open only when a researcher is present. The model also incorporates research on learning and assesses the effect of E-PPSR on social science learning experiences. Do museum visitors build greater conceptual understanding of the social science by contributing to real research and seeing their own responses within the larger dataset? Do they attend more deeply to debriefing activities when they have contributed their own data? The three main deliverables include: 1) a prototype Exhibit-based PPSR laboratory at the Exploratorium comprised of one exhibit for gaining informed consent, three 'Data-Catcher' exhibits modified to record anonymous responses when visitors opt-in to contributing to social psychological research, and one debriefing exhibit. A back-end database will send data to the academic researchers; 2) evaluation studies that test the E-PPSR model. The studies will assess the success of debriefing approaches, the effectiveness of recruitment and the impact of E-PPSR on learning. The team will publish a journal article describing the E-PPSR model and academic research findings. The team will also organize a conference session with others in the museum field who manage in-house academic research laboratories; and 3) a report by the academic partners describing the impact of the project on their research program.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
The Multi-Site Public Engagement with Science—Synthetic Biology (MSPES) initiative was an Innovations in Development project funded by the National Science Foundation (DRL-1421179) through the Advancing of Informal STEM Learning program (AISL). MSPES promoted public engagement with science (PES)—a model of mutual dialogue and learning between public and scientist audiences—through the creation and distribution of PES kits to nearly 200 informal science education sites around the country. Kits included two types of learning experiences: (1) forum programs during which scientists and teen or
This poster, which was presented at the Visitor Studies Association Conference in 2017, shares methods, working definitions and frameworks, and lessons-learned from Developing Guidelines for Designing Challenging and Rewarding Interactive Science Exhibits (DRL-1612577). Through design-based research, this project is investigating the emotional state of “productive struggle,” an intensive, fruitful state of exhilaration and challenge characterized by high levels of physiological arousal along with the experience of negative emotions. The project pushes beyond the positive range of emotional
Exploration Place, with funding from the National Aeronautics and Space Administration (NASA), contracted RK&A to conduct a summative evaluation of Design Build Fly, an exhibition and program series that explores what happens behind-the-scenes in Wichita’s aircraft plants. The goals of the study were to identify how visitors use the exhibition, explore what meaning visitors make from Design Build Fly, and understand to what extent visitors’ meaning-making aligns with intended outcomes.
How did we approach this study? RK&A conducted timing and tracking observations of visitors to the
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Museum of Science, Boston (MOS) and Boston University (BU) will conduct a Pilot and Feasibility Study project that leverages the current Living Laboratory (LL) model and expand it to engage high school students (teens) in experimental psychology research, science communication and science education activities. In LL, which is now an extensive network of museums and university researchers across the country, scientists and museum staff collaborate to engage children in studies on the museum floor and educate caregivers about the research. Multi-site implementation and evaluation of LL has also documented positive impacts for undergraduate researchers. Many sites are eager to extend these benefits to high school students by engaging them as practitioners within the model and by providing them with opportunities to engage in current research, education and communication, thereby helping to foster stronger youth identities with science and its applications in society. This project expands a ten-year LL partnership between MOS and BU to: 1) pilot a program in which high school students both conduct scientific research and engage the public in learning about science; 2) explore strategies for museums and universities to collaboratively engage, support and mentor high school students in science research, communication and education activities; 3) document curricular, other programmatic, and evaluation materials; and 4) convene professional participants to provide feedback on pilot materials, and assess the viability of implementing similar programs at additional sites. Guided by developmental evaluation, these activities will generate knowledge for the field, and act to increase professional capacity to integrate experiences for teens at multiple LL sites in future projects. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This workshop is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project will conduct an in-person participatory workshop designed to broaden participation in STEM. The project is designed to build the capacity of informal science institutions (ISIs) and their partners to use evidence-based collaborative project management practices and culturally responsive approaches to manage complex partnership efforts. The workshop will develop structure and provide tools for effective collaborative project management between ISIs including science centers and museums, and their research and community partners. To guide the workshops, participants will explore some core dimensions of culture to understand audiences, communities traditionally underserved by informal STEM institutions, and institutional differences. Participants will leave with concrete, actionable plans that will enable them to more effectively bring their collaboration plans to fruition. The project will develop resources including a playbook of project management templates, video vignettes, and summaries of relevant research findings and lessons from the field. Proceedings and resources from the workshop will be shared online with the field and ultimately archived on informalscience.org.
Conference participants will build the human resource expertise, infrastructure, and capacity needed to develop and maintain lasting partnerships that will further their institutions' aspirations to broaden participation in STEM. The project brings together an experienced, multidisciplinary team from informal science education and private industry professionals to advance knowledge of the skills and tools needed for managing complex collaborations between researchers, practitioners, and community stakeholders that will increase their effectiveness when working with diverse audiences. During the workshop, participants will think through their current project work by using a defined collaborative project management process to build a foundation of understanding and trust. Upon completion of the Institute, participants will have: (1) a shared purpose and goals that represent the interests of all parties; (2) identified some cross cultural and organizational differences that need to be accommodated for successful collaboration; (3) fostered joint decision making based on objective steps vs priorities of one organization; and (4) clarified deliverables of their project and how to measure success; (5)identified individual roles and responsibilities within their collaboration.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
In June of 2017, the Exploratorium hosted the GENIAL Summit which brought together approximately 91 practitioners, community leaders, diversity-focused organization staff, researchers, and media/marketing specialists from across the country and Puerto Rico for a one and a half day gathering. The National Science Foundation (NSF) funded Summit focused on increasing Latino participation in Informal STEM Learning (ISL) environments by examining these five content strands identified by the GENIAL team:
1. Latino Audiences
2. Marketing, Communicating, and Media
3. Community Collaboration and
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will build and test a new model for co-created public engagement with science activities in partnership with civic, community, and scientist partners. The innovation to be tested is deliberative dialogues in science museums that help reduce the polarization about socio-scientific issues, giving people a greater voice in science, and addressing barriers that disconnect scientists from the public. The project will engage four target audiences (informal science education/ISE professionals, civic, community and scientist partners). Science museum partners include Museum of Science (MOS) Boston, Oregon Museum of Science, the Michigan Science Center, and the North Carolina Museum of Life and Science. The project is designed to have a strategic impact on how ISE institutions choose topics of STEM engagement and build public Forum programs.
There will be two evaluation teams for the project. MOS Research and Evaluation will act as formative evaluation mentors for the four partner sites as they co-create their forums. They will provide evaluation capacity building for the sites using team-based inquiry as they create and understand the potential impacts and outcomes of the model. Data collection will include panel surveys and focus groups. The evaluation will explore how the forums can decrease 1) public polarization around STEM topics and (2) the disconnect between scientists, civic organization, and the public. The external summative evaluation will be conducted by Rockman et al (REA). They will conduct a study of the project's process to help the team identify challenges and strategies for overcoming them as they work through the phases of public engagement. The summative study will focus on the project goals taking a qualitative approach. Early interviews with partner participants will explore their strengths and weaknesses in taking on this type of public engagement model including the extent of their previous work with civic partners. Later interviews will investigate what factors have enabled or hindered this project. Summative evaluation questions will also address changes in attitudes toward public engagement with science. REA will collect feedback from summit attendees through intercept interviews and post-surveys administered within a week at the event's conclusion to explore the any changes in knowledge or confidence in undertaking this type of model. REA will present findings from the external evaluation during the annual meeting of the Association of Science-Technology Centers and publish reports to Informalscience.org. Once the model has been developed and tested it will be disseminated to an initial group of 25-30 science museums and eventually to the entire ISE museum field.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This set of appendices represent all research instruments related to study presented in the I/CaLL Art Experiences and Advancing Science Literacy report (NewKnowledge Publication #NSF.097.115.07).
Appendix A: Installation Site Intercept Interview Instrument
Appendix B: Artists-Scientists Walk & Talks Instrument
Appendix C: Post-Performance Event Survey Instrument
Note that researchers did not use an instrument for the fourth aspect of the study, the post-performance event reflection sessions. Instead, they allowed the discussions to be directed by the reflection participants.
DATE:
TEAM MEMBERS:
John FraserRebecca Joy NorlanderSophie SwansonNezam ArdalanKate FlinnerJoanna Laursen BruckerNicole LaMarca