Formative Evaluation: Sphere Corps Program

The Science Museum of Virginia (SMV) contracted with Randi Korn & Associates, Inc. (RK&A) to evaluate Sphere Corps, a Science on a Sphere program about climate change developed by SMV with funding from the National Oceanic and Atmospheric Administration (NOAA). The study was designed around RK&A's belief that organizations must be intentional in their practice by continually clarifying purpose, aligning practices and resources to achieve purpose, measuring outcomes, and learning from practice to strengthen ongoing planning and actions. To this end, the Sphere Corps project included five phases of work a literature review, a workshop to define intended program outcomes, two rounds of formative evaluation, and two reflection workshops. Formative evaluation data were collected using naturalistic observations and in-depth interviews. Each phase of work allowed staff to explore their vision for the Sphere Corps program and how it changed over time as they learned from and reflected on evaluation findings. What did we learn? SMV staff's goal was to create a facilitated, inquiry-based Science on a Sphere program about climate change. RK&A first completed a literature review that revealed a facilitated Sphere experience was in keeping with best practices and that using inquiry methods in a 20-minute program would be challenging but worth exploring further. Staff then brainstormed and honed the outcomes they hoped to achieve in Sphere Corps, which guided planning and script development. The first round of formative evaluation identified implementation barriers and an overabundance of iClicker questions, all of which created a challenging environment for educators to effectively use inquiry. Upon reflection, staff reduced the number of iClicker questions and added visualizations and questions that required close observation of the Sphere. Following a second round of formative evaluation, staff made additional changes to the program script and began to reflect on the reality of using inquiry in a single 20-minute program. Since the script covered a range of topics related to climate change, staff wondered if they should instead go deeper with one topic while encouraging more visitor observation and interpretation of Sphere data. Out of this discussion arose the idea of mini-programs a series of programs that would focus on communicating one key idea about climate change, such as helping people understand the difference between weather and climate. What are the implications of the findings? Central to the idea of the mini-program is the idea of doing less to achieve more. Impact and outcomes are incredibly difficult to achieve and trying to achieve too much often results in accomplishing very little. Through a reflection workshop and staff discussion, the SMV team was able to prioritize and streamline the outcomes and indicators originally written for the Sphere Corps program. Staff also recognized that their primary goal with the Sphere Corps program is to encourage visitors to think more critically about the science behind climate change. By scaling down the number of topics covered in the presentation, each program could intentionally focus on: (1) one key idea or question related to climate change; (2) achievement of only a few intended outcomes; and (3) implementation of specific facilitation strategies to achieve those outcomes. Intentionally covering less content also opens up opportunities to more effectively use inquiry methods. Included in the appendix of this report is the outcomes matrix used in the study.

If you would like to edit a resource, please email us to submit your request.