The most important consideration in evaluating chemistry outreach efforts is how to best use the evaluation to serve project needs. Evaluation should be about making programs more effective—at communicating ideas, changing attitudes, inspiring action, or reaching wider audiences, for example. A well-conducted evaluation typically contributes to the quality of a project by helping its leaders better define their goals, identify important milestones and indicators of success, and use evidence to support ongoing improvements. At its best, evaluation is an integral part of project design and
National Air and Space Museum, Washington DC
Summative Evaluation for Educational Program, 2016
The National Air and Space Museum (NASM) contracted Randi Korn & Associates, Inc. (RK&A) to conduct a summative evaluation of Science in Pre-K, an onsite and online teacher professional development program that NASM developed for the District of Columbia Public Charter Schools’ (DCPS). The program supports preschool teachers in teaching science through exploration and problem solving. This evaluation is a continuation of a planning and evaluation project initiated in 2013. In 2013, RK&A
SRI International and the Carnegie Foundation for Advancement of Teaching will jointly lead a workshop with the dual goals of supporting INCLUDES (Inclusion across the Nation of Communities of Learners that have been Underrepresented for Diversity in Engineering and Science) Launch Pilots during their first year and contributing to plans for a National Backbone. The workshop will strengthen and deepen the potential for rapid, continuous improvement cycles within and across INCLUDES networks. To do so, they will combine the expertise of SRI and Carnegie in improvement science, rapid analytics, and fostering networks of researchers and practitioners to engage participants in conducting a complete improvement cycle within and across INCLUDES Launch Pilots.
The workshop will have three phases. A first phase, conducted online, will share expertise related to each of the four parts of a complete improvement cycle: (1) problem definition, (2) data collection, (3) formative evaluation, and (4) effective communication. This first phase will combine an initial presentation with facilitated, online opportunities to interactively engage in the topics. A second phase, conducted face-to-face, will work intensively with teams from INCLUDES networks to improve their operational and long-term plans. The third phase will reflect and report on the workshop and contribute plans to build capacities for the National Backbone organization. Through the combination of these three phases, they will support the first-year work of INCLUDES teams and also refine understanding of how a national network could combine online and face-to-face elements to advance INCLUDES efforts. The workshop team will create and disseminate resources that are immediately useful to INCLUDES and related projects, and the workshop will openly coordinate with other workshops to achieve synergies. The online offerings will be open, broadly advertised, and permanently available. The lessons learned regarding plans for a national backbone will be disseminated broadly. In addition to participants from INCLUDES networks, additional stakeholders will be invited to both phases so as to shape the future plans to achieve broader impacts aligned with overall INCLUDES goals.
This evaluator reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It reflects on the nature of learning and challenges assumptions of outcomes from engaging in informal learning experiences.
This leadership reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It provides an organizational leadership perspective on evaluation.
This practitioner reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It discusses improving practice through reflection.
The Let’s Talk project examined what we know and don’t know about dialogue-based programs in museums. Through research, a symposium and an experimental graduate course, the project created a set of priorities and resources for moving dialogue work forward, developed new relationships across STEM based and cultural based museums and prepared a pool of pre-professionals to enter the field with knowledge of the value and potential of dialogue. The project ends with a call for further articulation and appropriate measurement of the intended and possible impacts and further development of field
The Museum of Science (MOS) will conduct a conference and associated activities to consider ways to foster the STEM workforce via both science research and science communications experiences for students at the high school level. The work will draw on and expand the scope in the USA of the NSF-funded National Living Laboratory Network which currently involves more than 350 institutions across 48 states and 21 countries. The National Living Laboratory initiative involves university researchers and museum professionals in co-implementing research and science communications activities in museum settings with the public, primarily families with young children. The research and communications focus is on cognitive science pertaining to the development of young children. While many scientists and museum professionals are interested in integrating high school research experiences into their practices, particularly for under-represented youth, existing infrastructure at museums and universities limits the quality of experiences and quantity of students that institutions can support. Current academic and museum members of the National Living Laboratory community have identified an opportunity to advance shared interests and knowledge in engaging youth in STEM by leveraging the Living Laboratory framework.
This project involves pre- and post- conference activities and will convene a group of science research and museum professionals at a workshop in Boston, MA to: 1) document current opportunities and challenges in engaging high-school aged youth in cognitive science research activities; 2) outline strategies to engage youth in research and science communication through Living Laboratory, with particular emphasis on cognitive sciences; and 3) create and disseminate a report on workshop outcomes through existing communication channels in both fields. The project includes pre-conference surveys of professionals about the topic and an evaluation of the project activities and outcomes. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Since its completion in 1937, the Golden Gate Bridge has become one of the world’s most recognized landmarks as both an iconic public works accomplishment and a popular tourist destination. In 2008, the National Science Foundation (NSF) awarded a $3 million grant to the Golden Gate Bridge Highway & Transportation District to leverage this status in developing informal education resources to interpret the science, engineering and history of the bridge. Through this initiative the Golden Gate Bridge would become a model for other public works venues for providing informal science education and
This summative evaluation report details the Broad Implementation of the Living Laboratory model--an initiative to promote partnership between museums and cognitive science researchers in order to promote professional learning and involve the public in scientific research. The evaluation investigated the extent of the dissemination effort’s depth, spread, sustainability, and shift in ownership, based on Coburn’s criteria for scale-up (2003). Evaluators collected data from surveys, interviews, focus groups, document review, and observations. Findings about depth suggest that adopters fully
The project will conduct a nation wide study to address three broad questions:
(1) How does the public view zoos and aquariums and how do these institutions affect STEM (Science Technology Engineering Mathematics) learning outside their walls?
(2) How do visitors experience zoos at different stages in their lives and how do zoo visits affect their knowledge and perspectives concerning environmental issues and conservation?
(3) What are the entry characteristics of visitors and how do those characteristics play out in behaviors during a visit?
The project is designed to advance understanding of how informal STEM learning emerges through the intersection of institutional pedagogy and learning goals and the characteristics of individuals and their social and cultural backgrounds. As the first institutional study that advances a field-wide research agenda, the project will map how to implement a national collaborative effort that can help refine program delivery and cooperation between zoos, aquariums and other STEM learning institutions.
The study will describe zoo and aquarium visitors based on a broad understanding of demographics, group, and individual perspectives to expand understanding of how these factors influence visitor learning and how they view the relevance of educational messages presented by zoos and aquariums. The project will result in reports, workshops and a handbook presenting findings of practical value for educators, a research platform and research tools, online discussion forums, and directions for future research. The project, led by New Knowledge Organization (NKO), will be carried out through the collaboration of NKO with other informal research organizations and the Association of Zoos and Aquariums (AZA) with its 230 informal science learning institutional members. This project is supported by the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings, as a part of its overall strategy to enhance learning in informal environments.