Year 5 Summative Evaluation of Exhibits and Programs

April 10, 2011 | Public Programs, Exhibitions

The NSF-funded Nanoscale Informal Science Education (NISE) Network produced exhibits and programs designed to develop awareness, engagement, and understanding of nanoscale science, engineering, and technology in the museum-going public. As part of the overall summative evaluation of the first five years of this grant, the Exhibits and Programs Study examines the measurable impacts of these public products on museum visitors. These exhibits and programs were developed during the first four years of the project as the NISE Network itself was growing and developing; the products show the strength of the network overall, and of its development of new ways to engage the public successfully in nano. Many of the gaps or missed opportunities identified in this report are already being addressed in the current work in Year 6 of this project; opportunities for further exploration are being considered by the Research team. The measurable impacts were separated into four main areas for the purpose of this study: enjoyment and interest; broader awareness of nanoscale science, engineering, and technology (often referred to as nano in this report); more specific content knowledge or understanding of nano; and perceived relevance of nano. These areas correspond to particular categories defined in the recent NRC-authored book, earning Science in Informal Environments (2009). This evaluation was focused on how visitors interact with the exhibits or programs in as realistic settings as possible. This required studying the exhibition in locations where it is currently on display (permanently in Boston, and in Portland Oregon, and traveling through a museum network in Arkansas), as a whole (not as individual pieces), and using regular, not cued, visitors. It also meant looking at programs as presented by typical presenters (trained volunteers or paid staff members, none of whom are experts in nano or were the developers of the program). These more realistic assessments required a more complex study, but one that more accurately reflects what a typical visitor might learn, wonder about, and connect to when seeing NISE Network exhibits or programs at a museum. Whenever possible, the evaluation included questions that reached beyond the exhibition to explore how people might use or think about nano in their lives beyond the walls of the museum. These more challenging sets of goals allow readers to see future possibilities of nano exhibits and programs; in many cases these were beyond the original goals of the project and should be seen as exploratory work, not as a report on the success of the Network. Key findings include:-Exhibits and programs effectively engage visitors with nano content Through the course of the project, the network has been successful at increasing the enjoyment and interest that visitors find in the exhibits and programs, with later versions outdoing earlier ones. Though visitors enter with low expectations for the nano topic, they rate the exhibits and programs as reasonably enjoyable and interesting - indicating that the Network has risen to meet a key challenge for this topic. Adult descriptions of engagement center on the nano subject matter itself; children are more likely to discuss interactive exhibit elements. -Visitors who see exhibits and programs show higher levels of nanoawareness Museum visitors rate their awareness of nano somewhat higher than the general public rates its awareness of nano. Visitors who see the nano exhibits and programs express higher confidence in their general nano knowledge than those who don't see the exhibits or programs; regression analysis suggests this difference is due to their time at the exhibition or program. -Many visitors associate nano with small, even before seeing nano in the museum In all adults groups that we sampled (at all sites, including those who had seen the Network products and those had not), at least 60% of visitors answered an open-ended prompt to explain what nanoscale science is about with a response that included the idea of small. This number was higher than anticipated, and only increased slightly among those who saw exhibits or programs, suggesting that (perhaps thanks to popular consumer products with nano in their title) the general public has developed a new definition of nano as meaning small, a definition that may or may not include scientific accuracy or convey associations having to do with nanoscale science, engineering, and technology. Exhibits and programs offer ways for visitors to deepen their nano knowledge We investigated visitors' understanding of nano in more depth by analyzing the definitions of nano they gave us, and by observing how they operationalized those definitions when sorting everyday objects into nano and non-nano groups. Visitors who saw the exhibition did not offer definitions of nano that differed in significant ways (they generally continued to use the small definition, and did not change in the level of accuracy present), but they did show more sophisticated understanding of nano as they sorted everyday objects. Visitors who saw the programs gave more accurate definitions of nano, with higher levels of scientific accuracy and a lower rate of alternative conceptions than their comparison group. (Program visitors did not complete interviews with an object-sorting activity and so their operational definitions could not be analyzed.)-Visitors find relevance in the exhibits and programs, and may find more ways to connect their everyday lives to nano when they encounter it in the future Visitors who have not seen exhibits or programs perceive nano has having less of a connection to their everyday lives than those who have seen Network products. When the NISE Network began, appropriate approaches for engaging the general public in this emerging area of science and technology were not clear, and success was not a given. After five years, it's clear that the Network has found successful approaches to initially engage the public on the museum floor, communicate important content and help visitors connect nano with their everyday lives, which may well allow those visitors to have more meaningful and sustained encounters with nano when they come across it in the future, in a world where nano is only increasing in ubiquity. As the Network continues to grow, this work (and the process used to produce it) provides a firm footing for further development of public products.