Informal learning, and by extension, museums, are inherently emotional experiences, evoking feelings of awe, excitement, and curiosity. Oftentimes, museum professionals have prioritized traditionally positive emotions such as excitement and interest as being the most desirable and useful in supporting museum learning. However, prior research into naturally occurring emotions at museums found that some visitors who experienced negative emotions, such as confusion or frustration, at exhibits also reported deeper engagement and overall feelings of satisfaction (Rappolt-Schlichtmann et al., 2017). Based on these findings and similar results from formal education (D’Mello et al., 2014), this project team has worked to develop and refine a framework of strategies for creating exhibits that invoke and support visitors through the complex emotional state called productive struggle (PS) which is defined as a three-part emotional arc characterized by: 1) disequilibrium (experienced emotionally as emotions like confusion, frustration, surprise, or unease) that arises from encountering a challenging task, phenomenon, or idea, 2) persistence through the task which is supported by exhibit design scaffolds, and 3) an emotionally productive resolution tied to the source of disequilibrium or an overall sense of effortful achievement. In deliberately attending to and supporting a range of negative and positive emotions in museums, visitors can gain access to a wider variety of complex emotional experiences, including those critical to STEM learning, and have potential to broaden participation in STEM by supporting learners' diverse emotional needs and preferences.
This multidisciplinary project team consisting of researchers and exhibit professionals utilized a design-based research (DBR) process to develop, test, and refine a definition of museum-based productive struggle, and create a framework of design strategies that support PS. Three physical exhibits and a virtual exhibit were created during this multi-year project. Additionally, a summative research study was conducted with 105 youth ages 10-17 to explore: 1) whether visitors experienced the expected emotional arc of PS; 2) how exhibit design strategies supported PS; and 3) the extent of visitors’ learning and engagement at these PS exhibits.
With support from NSF/AISL, the Exploratorium held the Generating Engagement and New Initiatives for All Latinos (GENIAL) Summit on June 5-6, 2017, in San Francisco, California.
The goals of the GENIAL Summit were to:
- Identify needs and opportunities for Latinos in informal science learning (ISL) environments.
- Facilitate and strengthen professional relationships.
- Identify recommendations and actionable insights with an outlook toward the future.
- Contribute to a more informed ISL field.
A total of 91 participants, a mix of practitioners, community leaders, media specialists, government officials, policy professionals, and researchers from across the United States and Puerto Rico participated in the Summit.
Summit outcomes will include: strengthened partnerships and new collaborations; examples of how participants of the GENIAL summit are applying the results of the gathering in their practice; and dissemination sessions, webinars, and post-summit documentation and articles with the results of GENIAL.
The purpose of this proposal is to convene scholars at a two-day conference to closely examine validity-related measurement issues, create a guidelines for the field of mathematics education research that addresses key ideas (e.g., validity, validity arguments, evidence for validity and measurement at-scale), and set a clear pathway for scholars to discuss quantitative measurement within mathematics education. Invitees will include a mix of young, older and diverse scholars in mathematics education research. Products include refereed journal articles along with a website.
The workshop will engage the Mathematics Education, Policy, Statistical, Psychometrics and other education research communities in examining and critiquing measurement validity evidence of mathematics education research with the long-term goal of increasing the quality of quantitative inference in mathematics education research (to include improvements in the training of doctoral students).
DATE:
-
TEAM MEMBERS:
Jonathan BosticMichele Carney
resourceprojectProfessional Development, Conferences, and Networks
The Center for Advancement of Informal Science Education (CAISE) is a National Science Foundation (NSF) funded resource center, working in cooperation with the NSF Advancing Informal STEM Learning (AISL) program to build and advance the informal STEM education field. CAISE continues the work it began in 2007--serving professional audiences in informal STEM learning, which includes those working in science centers and museums, zoos and aquariums, parks, botanical gardens and nature centers, events and festivals, libraries, making and tinkering spaces, media (TV, radio, film, social), cyberlearning and gaming, and youth, community, and out-of-school time programs.
What We Do:
CAISE seeks to characterize, highlight, and connect quality, evidence-based informal STEM learning work supported by a diversity of federal, local, and private funders by providing access to over 8,000 (and growing) resources that include project descriptions, research literature, evaluation reports and other documentation on the InformalScience.org website. In addition, CAISE convenes inquiry groups, workshops and principal investigator meetings designed to facilitate discussion and identify the needs and opportunities for informal STEM learning.
In this award, CAISE is also tasked with advancing and better integrating the professional fields of informal STEM learning and science communication by (1) broadening participation in these fields, (2) deepening links between research and practice, and (3) building capacity in evaluation and measurement. These activities are being undertaken by cross-sector task forces of established and emerging who will be responsible for conducting field-level analyses, engaging stakeholders, and creating roadmaps for future efforts. CAISE is also building on existing communication channels for dissemination to the larger field, and through the InformalScience.org website. An External Review Board and Inverness Research are providing oversight of CAISE's program activities and evaluation of the center.
Who We Are:
CAISE operates as a network of core staff housed at the Association of Science-Technology Centers (ASTC) in Washington, D.C. and co-principal investigators and other collaborators at academic institutions and informal STEM education (ISE) organizations across the U.S. Other key collaborators are the American Association for the Advancement of Science's Center for Public Engagement with Science, the National Informal STEM Education Network, and Arizona State University.
The Orlando Science Center (OSC) in partnership with BASE Camp Children's Cancer Foundation and the University of Central Florida (UCF) will engage chronically ill children (cancer, sickle-cell, HIV/AIDS, etc.) and other Orlando area youth ages 10-18 who face the greatest educational disparities in NASA-themed Aeronautics, Space Exploration, and Space Science exhibits through a STEM engagement and educator professional development project entitled STEM Satellites: A Mobile Mathematics and Science Initiative for Orlando Metropolitan Area Children's Hospitals. OSC will partner with educational researchers, evaluators, and planetary scientists from the University of Central Florida to create three mobile exhibits for each of the three children's hospitals in the Orlando metropolitan area. Two additional sets of the three mobile carts will be used at OSC and UCF. The three mobile exhibits will be based on the planned NASA missions that the UCF planetary scientists are leading including a Mars-themed exhibit focusing on space exploration, an asteroid-themed exhibit, and an exhibit on microgravity. Each cart will include multiple STEM activities that incorporate NASA data and artifacts from prior NASA missions, UCF planetary science collections, and Kennedy Space Center. OSC will provide professional development and training to BASE Camp volunteers who will supervise the use of the mobile exhibits in the hospitals. These exhibits will provide authentic experiences that mirror current and planned NASA missions at a level that the children can understand. These hands-on and engaging exhibits will not only help motivate children to pursue STEM careers but will also help educate the general public about the exciting and important work that NASA carries out. Providing this level of engaging and authentic STEM activities through the mobile exhibits to this historically underrepresented population is unprecedented.
DATE:
-
TEAM MEMBERS:
JoAnn NewmanJosh ColwellBrandan LanmanMegan Nickels
The Bronx Museum of the Arts will hire an evaluator to asses and facilitate the growth of its three signature programs–Bronx Stories, Back in the Bronx and Bronx Lab–to ensure that they continue to celebrate, affirm, and advance the vibrancy of the borough and its multigenerational voices. The museum will assess training, seek community input, and apply new knowledge gained through program redesign and implementation. It will enhance outreach and communication, and analyze and share outcomes. Using an advisory group of community stakeholders, the museum will amplify its services by drawing increased audiences, expanding its role as a cultural gateway, and highlighting the Bronx as a dynamic community.
Portal to the Public: Expanding the National Network (PoP: ENN) is implementing around the county the successful NSF-funded Portal to the Public model in which researchers are trained to communicate and interact with the general public at informal science education (ISE) institutions about the research that they are conducting. The project, which follows on a thorough evaluation of the model at eight sites and current implementation at an additional fifteen sites, will incorporate twenty new ISE sites into the growing network, provide training and mentorship to ISE professionals on the use and adaptation of the PoP implementation manual and toolkits, and develop an enhanced network website that will serve as a communication and innovation hub. The work is responsive to the needs and activities of ISE organizations which continue to expand their missions beyond presenting to the public established science, technology, engineering and math (STEM) and are working to become places where visitors can also experience the process and promise of current research via face-to-face interactions with researchers. The project is expanding both the kind and number of institutions involved around the country and is facilitating their capacity to develop a knowledge base, share experiences and best practices.
This project is making novel use of familiar technology (smartphones and tablets) to address the immediate and pressing challenge of affordable, ongoing, large-scale museum evaluation, while encouraging museum visitors to engage deeply with museum content. Using a smartphone app, museum visitors pose questions to a 'virtual scientist' called Dr. Discovery (Dr. D). Dr. D provides answers and the chance to complete fun mini-challenges. The questions visitors ask are gathered in a large database. An analytics system analyzes these data and a password-protected website provides continuous, accessible evaluation data to museum staff, helping them make just-in-time tweaks (or longer term changes) to exhibit-related content (such as multimedia, lecture topics, docent training, experience carts, etc.) as current events and visitors' needs and interests change. The intellectual merit of this project is that it is building evaluation capacity among informal educators, advancing the fields of visitor studies, museum evaluation, informal science learning, and situated engagement, and is contributing to the development of novel evaluation techniques in museums. This project has many broader impacts: The Ask Dr. Discovery system is available to any venue that wishes to use or adapt it to their context. By enhancing the visitor experience and improving museum access to data for evaluation and data-driven decision making across the country, Ask Dr. Discovery has both a direct and indirect impact on museums and visitors of all types. This project is also training the next generation of STEM and education innovators by employing a diverse team of undergraduate students.
EvalFest (Evaluation Use, Value, and Learning through Festivals of Science and Technology) will test innovative evaluation methods in science festivals that are being held across the country and assess in what ways and how effectively they are used. Morehead Planetarium and Science Center (at the University of North Carolina-Chapel Hill) and the University of California, San Francisco, in collaboration with over twenty science festivals, will (1) investigate whether a multisite evaluation approach is an effective model for creating common metrics for informal STEM education, (2) develop common methods to measure the effects of Festivals, (3) create a query-able database of 50,000 Festival attendees to share with the informal STEM learning field, and (4) document whether these efforts also result in new knowledge related to informal STEM education. The project will develop the Enterprise Feedback Management (EFM) system and query-able database for the festival community. EFMs are systems, including processes and software, that enable groups (such as the festival network) to collect, organize, analyze and share data. The EFM system will be designed to integrate data across sites and to allow users to extract data of interest. The project will refine evaluation tools currently used within the Science Festival Alliance that assess self-reported festival learning, and the effects of festival attendance, motivation, and future science participation. It will collect economic impact data and longitudinal festival attendee data. The project will also develop some new evaluation tools such as secret shopper observational protocols. Data from festival attendees will be collected onsite at participating festivals.
This early-stage design and development, integrated media and research project will contribute important new understandings to the informal science learning literature by exploring science engagement on social media when integrated with broadcast television. It will help answer questions including: What does such engagement look like? Who participates? How and why does it happen? and What is the degree or depth of engagement? The project builds on the previous successful work by WGBH nationally distributing the television series NOVA scienceNOW and the research expertise of EDC. WGBH's NOVA scienceNOW program will collaborate with EDC to develop new metrics to understand how and why learners engage with science on social media. Deliverables will include six one-hour episodes of NOVA scienceNOW, short online videos, moderated online discussion events, and an online film festival. A new social Media Initiative will develop six live broadcast microblogging events, six post-broadcast online discussion events, daily social media updates, and an online film festival that will feature user generated videos. A range of STEM content in the videos and online posts will be framed around big science and engineering questions such as animal communication and survival systems, the biology of sleep, climate change, new technologies, energy, genetics, and natural disasters. The continued innovations and expansion of social media channels provides significant new opportunities for providing learner's access to high quality science content, researchers, and opportunities to participate in science. In the first phase of this work to deepen the evidence based understanding of how social media supports informal science engagement, NOVA and EDC will collaborate to develop new measurement instruments: (1) a Network Profile to quantitatively represent the size and activity of NOVA's social media network; (2) an Informal Science Engagement (ISE) index to measure the degree of engagement by coding and analyzing conversations and posts; (3) a Follower Profile to assess the degree of activity and the nature of the engagement; and (4) a Science Social Media Engagement survey instrument. They will then use these measures and data collection protocols to explore whether and how the initiative might influence science engagement. External expert reviewers with content and methodological expertise will review all aspects of the project at critical junctures. This project will contribute important new knowledge and research instruments and methods to better understand how the learning opportunities of social media channels can be realized most effectively. This has significant potential for broad and lasting benefits to society as well as advancing the informal science learning field.
For over two decades NSF has been investing in the development and evaluation of giant screen films for viewing by audiences in science centers and museums. These have been highly successful in terms of audiences reached and project evaluations that indicate their impact on learning. Less well understood is how the unique attributes of giant screen films (e.g., "immersion" and "presence") affect learners in ways that differ from other film formats. This integrated research and media project will contribute to that knowledge base. Project deliverables will include a giant screen film that tells the story of the discovery of biological mimicry (the critical proof for natural selection and in turn, evolution) through the life story of Henry Bates and his travels through the Amazon rainforest more than 150 years ago; 2D dome, and 2D flat format versions; live interactive science demonstrations and educational resources; and workshops for ISE professionals. The film and the related outreach via science centers, social media, and the web are expected to reach large public audiences; workshops and web resources will reach ISE professionals nationally. A strategy for reaching underrepresented audiences through science museums and partnerships with educational societies is a part of the broadening participation effort. Building on results of an NSF-funded workshop in which researchers, evaluators, and filmmakers began to develop a research agenda to provide evidence about giant screen attributes and their impacts on learning, the research component of this project will focus on the differences in learner knowledge among the various film formats, their unique attributes, and whether format plays a role in science interest and science identity. A baseline study will be conducted to begin gathering evidence on how each of these formats affects learning. Data on audience knowledge gains, interest, and science identity will be collected using a novel tablet-based game-like assessment pre-film viewing, immediately post viewing, and in a later follow-up. These baseline data will inform follow-on research that, over time, can better explain the unique impacts on learning of the giant screen format. Project partners include the Pacific Science Center, SK Films, Howard Hughes Medical Institute, Rutgers University, and Arizona State University.
Indianapolis / City as Living Laboratory (I/CaLL) is a city-wide civic collaboration engaging in cross-sector research that builds on prior research in informal science learning in public settings. It extends research in place-based and service learning traditions, and uses the city itself as an informal science learning (ISL) environment for Science and Engineering for Environmental Sustainability learning outcomes. This project is creating place-based science learning experiences as part of public spaces in Indianapolis and establishes the next generation of urban science museums that increase opportunities for learning. The project will develop a self-sustaining program for art/science collaborations as they inaugurate city-sanctioned changeable installations at I/CaLL sites. Data from the project will be used to inform ISL professionals at museums throughout the community and around the country. Thousands of volunteers and their families will help create I/CaLL spaces, engage with communities, and serve as research participants connecting with science learning through site development. The unprecedented scale of this project provides a full measure of informal science service learning at a city scale, offering data that can change how science learning is measured, how people from all walks of life develop science literacy as part of their social public experience, and embodying the concept of the city as a living science learning lab. Broader impacts include the development of the city as an informal science learning environment that will become a new standard for thinking about what cities as cultural units can become as we build a resilient Science and Engineering culture for Environmental Sustainability.