The starting point of the Evaluating Evaluation project was our impression that despite the substantial resources that are spent on the summative evaluation of museums and galleries the research has little impact and largely remains ineffectual.
With support from the Wellcome Trust and the Heritage Lottery Fund, we set out to see why this seems to be the case and to explore whether there are ways in which the impact of summative evaluations on knowledge and practice might be enhanced. To this end, we reviewed a substantial number of reports, undertook a range of interviews and held two
DATE:
TEAM MEMBERS:
Maurice DaviesChristian Heath
resourceevaluationProfessional Development, Conferences, and Networks
The Summative Study of the Nano Mini-exhibition took place during the spring and summer of 2012. After being observed during their Mini-exhibition experience, 455 visitors across six different partner institutions participated in surveys and interviews with NISE Net evaluation team members. This report begins by describing the key findings of the study in detail, with additional information about study methods, instruments, and two exploratory sub-studies found in the Appendices.
In late 2012, Providence Children’s Museum began a major three-year research project in collaboration with The Causality and Mind Lab at Brown University, funded by a grant from the National Science Foundation (1223777). Researchers at Brown examined how children develop scientific thinking skills and understand their own learning processes. The Museum examined what caregivers and informal educators understand about learning through play in its exhibits and how to support children’s metacognition – the ability to notice and reflect on their own thinking – and adults’ awareness and appreciation of kids’ thinking and learning through play. Drawing from fields like developmental psychology, informal education and museum visitor studies, the Museum’s exhibits team looked for indicators of children’s learning through play and interviewed parents and caregivers about what they noticed children doing in the exhibits, asking them to reflect on their children’s thinking. Based on the findings, the research team developed and tested new tools and activities to encourage caregivers to notice and appreciate the learning that takes place through play.
This pathways project will study how audiences in public spaces, in this case those in a museum setting, relate to and make sense of large data displays. The project is preliminary to development of a traveling, hands-on exhibition enabling users to create and utilize representations of big data displays such as maps and charts. As the test case, the project will use science maps that provide an overview of science generally and specific areas of STEM, charting and exploring the history and future of science and technology. The data collection portion of the project will take place at the New York Hall of Science, the Marian Koshland Science Museum, COSI in Columbus, Ohio, and WonderLab Museum in Bloomington, Indiana. The project will create a foundation for the design of museum exhibits and educational programs that teach museum visitors how to explore, engage and make better sense of big data. The project is potentially transformative because big data is becoming ubiquitous and making sense out of large data displays is necessary in order to understand big data sets.
The Oregon Museum of Science and Industry is implementing a Pathways project that will test and refine a model to promote an appreciation that science is everywhere and personally relevant by engaging transit riders in Portland, OR with location-relevant STEM content through unfacilitated, interactive science exhibits in everyday places. The study will employ a "design-based research" approach that both iteratively tests exhibit effectiveness and develops and refines an underlyting theoretical model that can contribute knowledge to the field. The "Science on the Move" model will be developed and tested using an exhibit prototype that includes 1) an easily transportable prototype core with a familiar touch-screen interface, 2) multiple sets of interactive digital content, and 3) a variety of accompanying outer skins designed to attract the public. The exhibit prototype will be placed at bus transit nodes to reach adults, specifically targeting those without college degrees who are underrepresented in science centers. A range of possible STEM content domains will be selected and tested based on topics of interest to the public. If successful, given the several challenges involved, the feasibility study will be applied more fully around Portland and be a model for other cities to consider.
The University of California, Davis Tahoe Environmental Research Center (TERC), UC Davis W.M. Keck Center for Active Visualization in the Earth Sciences (KeckCAVES), ECHO Lake Aquarium and Science Center (ECHO), UC Berkeley Lawrence Hall of Science (LHS), and the Institute for Learning Innovation (ILI) will study how 3-D visualizations can most effectively be used to improve general public understanding of freshwater lake ecosystems and Earth science processes through the use of immersive three-dimensional (3-D) visualizations of lake and watershed processes, supplemented by tabletop science activity stations. Two iconic lakes will be the focus of this study: Lake Tahoe in California and Nevada, and Lake Champlain in Vermont and New York, with products readily transferable to other freshwater systems and education venues. The PI will aggregate and share knowledge about how to effectively utilize 3-D technologies and scientific data to support learning from immersive 3-D visualizations, and how other hands-on materials can be combined to most effectively support visitor learning about physical, biological and geochemical processes and systems. The project will be structured to iteratively test, design, and implement 3-D visualizations in both concurrent and staggered development. The public will be engaged in the science behind water quality and ecosystem health; lake formation; lake foodwebs; weather and climate; and the role and impact of people on the ecosystem. A suite of publicly available learning resources will be designed and developed on freshwater ecosystems, including immersive 3-D visualizations; portable science stations with multimedia; a facilitator's guide for docent training; and a Developer's Manual to allow future informal science education venues. Project partners are organized into five teams: 1) Content Preparation and Review: prepare and author content including writing of storyboards, narratives, and activities; 2) 3-D Scientific Visualizations: create visualization products using spatial data; 3) Science Station: plan, design, and produce hands-on materials; 4) Website and Multimedia: produce a dissemination strategy for professional and public audiences; 4) Evaluation: conduct front-end, formative, and summative evaluation of both the 3-D visualizations and science activity stations. The summative evaluation will utilize a mixed methods approach, using both qualitative and quantitative methods, and will include focus groups, semi-structured interviews, web surveys, and in-depth interviews. Leveraging 3-D tools, high-quality visual displays, hands-on activities, and multimedia resources, university-based scientists will work collaboratively with informal science education professionals to extend the project's reach and impact to an audience of 400,000 visitors, including families, youth, school field trip groups, and tourists. The project will implement, evaluate, and disseminate knowledge of how 3-D visualizations and technologies can be designed and configured to effectively support visitor engagement and learning about physical, biological and geochemical processes and systems, and will evaluate how these technologies can be transferred more broadly to other informal science venues and schools for future career and workforce development in these critical STEM areas.
Researchers at the American Association of Variable Star Observers, the Living Laboratory at the Boston Museum of Science, and the Adler Planetarium are studying stereoscopic (three-dimensional or 3D) visualizations so that this emerging viewing technology has an empirical basis upon which educators can build more effective informal learning experiences that promote learning and interest in science by the public. The project's research questions are: How do viewers perceive 3D visualizations compared to 2D visualizations? What do viewers learn about highly spatial scientific concepts embedded in 3D compared to 2D visualizations? How are viewers\' perceptions and learning associated with individual characteristics such as age, gender, and spatial cognition ability? Project personnel are conducting randomized, experimental mixed-methods research studies on 400 children and 1,000 adults in museum settings to compare their cognitive processing and learning after viewing two-dimensional and three-dimensional static and dynamic images of astronomical objects such as colliding galaxies. An independent evaluator is (1) collecting data on museum workers' and visitors' perceived value of 3D viewing technology within museums and planetariums and (2) establishing a preliminary collection of best practices for using 3D viewing technology based on input from museum staff and visitors, and technology creators. Spatial thinking is important for learning many domains of science. The findings produced by the Two Eyes, 3D project will researchers' understanding about the advantages and disadvantages of using stereoscopic technology to promote learning of highly spatial science concepts. The findings will help educators teach science in stereoscopic ways that mitigate problems associated with using traditional 2D materials for teaching spatial concepts and processes in a variety of educational settings and science content areas, including astronomy.
This summative evaluation study examines visitors' experiences of the "Plastics Unwrapped" exhibit at the Burke Museum of Natural History, Seattle, WA. The exhibit explores the complicated legacy of plastic, and the ways in which it has improved life, but not without serious impact on people and the environment. Within a framework of four evaluation questions, this study used multiple methods to assess what visitors do and where they spend their time in the exhibit, what knowledge they take away, and whether the exhibit impacts visitors' attitude toward plastic and their perception of the
COSI, in partnership with WOSU @ COSI, will be going forward with a project in which enhancements and other changes may be made to the WOSU exhibition space, entrance area, and adjacent hallway. This project may include, but is not limited to, introducing more elements of the PBS Kids brand, such as Sesame Street and Sid the Science Kid, into the exhibition space, introducing interactive elements regarding TV Production to the site, and adding loose parts to the Chroma Key exhibit. To inform decisions about the type and nature of enhancements most needed in the exhibit area, COSI desires to
Magnolia Consulting, LLC conducted a formative and summative evaluation to examine public perceptions of the utility and quality of two labs/exhibits within the North Carolina Museum of Natural Sciences Natural World Investigate Lab, Biofuels and Science of Scent. Appendix includes survey.
DATE:
TEAM MEMBERS:
North Carolina State Museum of Natural ScienceMary Styers
This report contains findings from a summative evaluation study of a set of four featured elements that comprise the new Nature Research Center of the North Carolina Museum of Natural Sciences. The elements were: 1) an exhibit, Ancient Fossils, New Discoveries; 2) Investigate Labs; 3) Daily Planet Scientist Talks; 4) Science Cafes. Evaluation was conducted as four distinct, multi-method studies to provide targeted understanding of visitor outcomes and experiences at each element. Findings across the four elements indicate that the features of the NRC are enjoyed by visitors and each supports
Research in experimental and developmental psychology, cognitive science, and neuroscience, suggests that tool fluency depends on the merging of perceptual and motor aspects of its use, an achievement we call perceptuomotor integration. We investigate the development of perceptuomotor integration and its role in mathematical thinking and learning. Just as expertise in playing a piano relies on the interanimation of finger movements and perceived sounds, we argue that mathematical expertise involves the systematic interpenetration of perceptual and motor aspects of playing mathematical
DATE:
TEAM MEMBERS:
Ricardo NemirovskyMolly KeltonBohdan Rhodehamel