In June of 2017, the Exploratorium hosted the GENIAL Summit which brought together approximately 91 practitioners, community leaders, diversity-focused organization staff, researchers, and media/marketing specialists from across the country and Puerto Rico for a one and a half day gathering. The National Science Foundation (NSF) funded Summit focused on increasing Latino participation in Informal STEM Learning (ISL) environments by examining these five content strands identified by the GENIAL team:
1. Latino Audiences
2. Marketing, Communicating, and Media
3. Community Collaboration and
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will build and test a new model for co-created public engagement with science activities in partnership with civic, community, and scientist partners. The innovation to be tested is deliberative dialogues in science museums that help reduce the polarization about socio-scientific issues, giving people a greater voice in science, and addressing barriers that disconnect scientists from the public. The project will engage four target audiences (informal science education/ISE professionals, civic, community and scientist partners). Science museum partners include Museum of Science (MOS) Boston, Oregon Museum of Science, the Michigan Science Center, and the North Carolina Museum of Life and Science. The project is designed to have a strategic impact on how ISE institutions choose topics of STEM engagement and build public Forum programs.
There will be two evaluation teams for the project. MOS Research and Evaluation will act as formative evaluation mentors for the four partner sites as they co-create their forums. They will provide evaluation capacity building for the sites using team-based inquiry as they create and understand the potential impacts and outcomes of the model. Data collection will include panel surveys and focus groups. The evaluation will explore how the forums can decrease 1) public polarization around STEM topics and (2) the disconnect between scientists, civic organization, and the public. The external summative evaluation will be conducted by Rockman et al (REA). They will conduct a study of the project's process to help the team identify challenges and strategies for overcoming them as they work through the phases of public engagement. The summative study will focus on the project goals taking a qualitative approach. Early interviews with partner participants will explore their strengths and weaknesses in taking on this type of public engagement model including the extent of their previous work with civic partners. Later interviews will investigate what factors have enabled or hindered this project. Summative evaluation questions will also address changes in attitudes toward public engagement with science. REA will collect feedback from summit attendees through intercept interviews and post-surveys administered within a week at the event's conclusion to explore the any changes in knowledge or confidence in undertaking this type of model. REA will present findings from the external evaluation during the annual meeting of the Association of Science-Technology Centers and publish reports to Informalscience.org. Once the model has been developed and tested it will be disseminated to an initial group of 25-30 science museums and eventually to the entire ISE museum field.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This project explores augmented reality (AR) technology as a way to make STEM content accessible to Deaf and Hard-of-Hearing (DHH) learners in "live" presentation settings found within science centers and museums. The current ways of providing communication and information access to DHH museum visitors require DHH learners to split their attention between the visual focus of the experience and the interpreter or captioning display. As a result, DHH learners are forced to decide which information to miss during live presentations, thus presentations may not be fully comprehended by the DHH learner. Furthermore, this issue perpetuates a lack of inclusivity and accessibility in informal STEM learning environments. The work focuses on investigating approaches to develop an AR platform with a near-eye display that will allow DHH visitors to receive signed or captioned instruction while still looking at and interacting with the intended exhibits. The new platform will allow for transmission of live and spontaneous instruction. Researchers will evaluate and recommend efficient ways to make information at science centers and museums accessible to the DHH population based on data collected from DHH learners. This project is funded by the Advancing Informal STEM Learning program's Pilot and Feasibility Studies which seek new approaches to design and development of STEM learning to be accessible to all learners including underrepresented groups in informal environments.
The researchers will investigate the following research question: how can the use of Augmented Reality (AR) as a method of support service delivery improve the engagement of Deaf and Hard-of-Hearing (DHH) adolescent learners in semi-structured learning environments? Both formative and summative evaluations will be conducted from two different angles: (1) an engineering-centric evaluation focused on technical development and (2) an educational-research centric evaluation focused on pilot studies on user experiences, attitudes, and learning outcomes. Both qualitative and quantitative data on the use of this technology will be collected. The evaluation methods include attitudinal surveys (pre/post), ethnographic observations, pre/post tests of content knowledge, and semi-structured group and individual interviews. This project will be conducted by researchers in the Center on Access Technology at National Technical Institute for the Deaf, one of the colleges of Rochester Institute of Technology, in partnership with the Rochester Museum & Science Center, which includes the Rochester Challenger Learning Center. This project represents a first step in setting future directions for research & development and to make educational materials more accessible to the DHH community.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Wendy DannelsSara SchleyBrian TragerMel Chua
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understanding of deeper learning by participants. This pilot study, Akeakamai (Hawaiian, literally lover of wisdom, scientist, scholar), will explore the convergence of contemporary Western science topics with indigenous Hawaiian culture-based science experiences as a mechanism to strengthen STEM perceptions, cross-cultural science collaboration, and multi-generational community engagement with STEM. The work is grounded in the notion that STEM learning within the context of local informal indigenous community settings should be culturally responsive and culturally sustaining, and should privilege indigenous epistemologies. If successful, the results of this pilot could provide valuable insights on effective approaches to developing and implementing culturally consistent and sustainable multigenerational STEM engagement among Native Hawaiians and Pacific Islanders, and across the Pacific region.
Over a two-year duration, the study will address three research questions. (1) To what extent does inclusion of culture into curriculum designed for informal Culture-Science Explorations mitigate perceived barriers to participation in science? (2) What barriers do community members perceive to limit their participation in science? (3) What are the areas of consonance between Native Hawaiian and Western scientific approaches to knowledge and learning? Approximately 200 predominantly Native Hawaiian and Pacific Islanders, ranging in age from 8 - 85 years old, will participate in the pilot. The research team will collect participant data during all phases of the social intervention, a suite of culture-science exploration experiences held at the Halau Inana, a Native Hawaiian community collaboration space. The intervention will employ pedagogical methods that are responsive to Hawaiian cultural norms to deliver content that integrates across the interfaces of Western science and technology and indigenous knowledge, and incorporates Hawaiian language. A rigorous external evaluation will also be conducted. The results of the research and evaluation will be broadly disseminated. Ultimately, the project aims to develop a conceptual and practical cross-cultural, multi-generational framework for community-based science learning in Hawai'i that can serve as a model for future research and programs that extend into and beyond indigenous communities of the Pacific region.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Helen TurnerJonathan BakerChrystie Naeole
resourceprojectProfessional Development, Conferences, and Networks
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project is a two-day conference, along with pre- and post-conference activities, with the goal of furthering the informal science learning field's review of the research and development that has been conducted on data visualizations that have been used to help the public better understand and become more engaged in science. The project will address an urgent need in informal science education, providing a critical first step towards a synthesis of research and technology development in visualization and, thus, to inform and accelerate work in the field in this significant and rapidly changing domain.
The project will start with a Delphi study by the project evaluator prior to the conference to provide an Emerging Field Assessment on data visualization work to date. Then, a two-day conference at the Exploratorium in San Francisco and related activities will bring together AISL-funded PIs, computer scientists, cognitive scientists, designers, and technology developers to (a) synthesize work to date, (b) bring in relevant research from fields outside of informal learning, and (c) identify remaining knowledge gaps for further research and development. The project team will also develop a website with videos of all presentations, conference documentation, resources, and links to social media communities; and a post-conference publication mapping the state of the field, key findings, and promising technologies.
The initiative also has a goal to broaden participation, as the attendees will include a diverse cadre of professionals in the field who contribute to data visualization work.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This set of appendices represent all research instruments related to study presented in the I/CaLL Art Experiences and Advancing Science Literacy report (NewKnowledge Publication #NSF.097.115.07).
Appendix A: Installation Site Intercept Interview Instrument
Appendix B: Artists-Scientists Walk & Talks Instrument
Appendix C: Post-Performance Event Survey Instrument
Note that researchers did not use an instrument for the fourth aspect of the study, the post-performance event reflection sessions. Instead, they allowed the discussions to be directed by the reflection participants.
DATE:
TEAM MEMBERS:
John FraserRebecca Joy NorlanderSophie SwansonNezam ArdalanKate FlinnerJoanna Laursen BruckerNicole LaMarca
Engagement is the cornerstone of learning in informal science education. During free-choice learning in museums and science centers, visitor engagement shapes how learners interact with exhibits, navigate through exhibit spaces, and form attitudes, interests, and understanding of science. Recent advances in multimodal learning analytics are creating novel opportunities for expanding the range and richness of measures of visitor engagement in free-choice settings. In particular, multimodal learning analytics offer significant potential for integrating multiple data sources to devise a composite picture of visitors' cognitive, affective, and behavioral engagement. The project will center on providing a rich empirical account of meaningful visitor engagement with interactive tabletop science exhibits among individual visitors and small groups, as well as uncovering broader tidal patterns in visitor engagement that unfold across exhibit spaces. A key objective of the project is creating models and practitioner-focused learning analytic tools that will inform the best practices of exhibit designers and museum educators. This project is funded by the Advancing Informal STEM Learning (AISL) program. As part of its overall strategy to enhance learning in informal environments, AISL funds research and innovative approaches and resources for use in a variety of settings. The research team will conduct data-rich investigations of visitors' learning experiences with multimodal learning analytics that fuse the rich multichannel data streams produced by fully-instrumented exhibit spaces with the data-driven modeling functionalities afforded by recent advances in machine learning and educational data mining. The research team will conduct a series of visitor studies of naturalistic engagement in solo, dyad, and group interactions as visitors explore interactive tabletop science exhibits. The studies will utilize eye trackers to capture visitors' moment-to-moment attention, facial expression analysis and quantitative field observations to track visitors' emotional states, trace logs generated by exhibit software, as well as motion-tracking sensors and coded video recordings to capture visitors' behavioral interactions. The studies will also use conversation recordings and pre-post assessment measures to capture visitors' science understanding and inquiry processes. With these multimodal data streams as training data, the research team will use probabilistic and neural machine learning techniques to devise learning analytic models of visitor engagement. The project will be conducted by a partnership between North Carolina State University and the North Carolina Museum of Natural Sciences. The research team will 1) design a data-rich multimodal visitor study methodology, 2) create the Visitor Informatics Platform, a suite of open source software tools for multimodal visitor analytics, and 3) launch the Multimodal Visitor Data Warehouse, a curated visitor experience data archive. Together, the multimodal visitor study methodology, the Visitor Informatics Platform, and the Multimodal Visitor Data Warehouse will enable researchers and practitioners in the informal science education community to utilize multimodal learning analytics in their own informal learning environments. It is anticipated that the project will advance the field of informal STEM learning by extending and enriching measures of meaningful visitor engagement, expanding the evidence base for visitor experience design principles, and providing learning analytic tools to support museum educators. By enhancing understanding of the cognitive, affective, and behavioral dynamics underlying visitor experiences in science museums, informal science educators will be well-positioned to design learning experiences that are more effective and engaging. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
James LesterJames MinogueJonathan RoweNorth Carolina Museum of Natural Sciences
resourceresearchProfessional Development, Conferences, and Networks
Leading science educators from 9 South and Southeastern Asian countries and the U.S. met for three days in Kuala Lumpur, Malaysia (October 4-6, 2017) in an effort to rethink and re-envision science education in the 21st Century. The attendees of this U.S. National Science Foundation-funded international conference reaffirmed the G8-Science Academies Joint Statement (2011) that education in science must be targeted not only to future scientists, engineers and other specialists in government and industry, but also to the general public, including school-aged children and adults. The attendees at
The Association of Science-Technology Centers (ASTC) aims to help member institutions inform and educate the public about science through a global lens. The World Biotech Tour (WBT) was a three-year initiative designed to not only promote a greater understanding of biotechnology through public outreach and programming led by science centers and museums, but also as an opportunity for ASTC to collect useful data on the resources required and challenges encountered at different science museums around the world when implementing a long-term, multi-country program. The WBT involved students
The San Diego Natural History Museum contracted RK&A to conduct an evaluation of the SPECTRUM Social Stories Project. The program involved a series of 16 “meetups” for museum staff and young adults with an autism spectrum disorder (ASD) to visit seven Balboa Park museums. The young adults, with support from museum staff, used their observations of those spaces to create pre-visit materials for museum-goers with ASD.
How did we approach this study?
The evaluation explored outcomes for young adult participants, museum staff, and project partners from other Balboa Park museums. We conducted
Supported by the National Science Foundation, the Global Soundscapes! Big Data, Big Screens, Open Ears project employs a variety of informal learning experiences to present the physics of sound and the new science of soundscape ecology. The interdisciplinary science of soundscape ecology analyzes sounds over time in different ecosystems around the world. The major components of the Global Soundscapes project are an educator-led interactive giant-screen theater show, group activities, and websites. All components are designed with both sighted and visually impaired students in mind. Multimedia