KQED QUEST Final Evaluation Report

KQED's QUEST is a multi-year, multiple-media project seeking to influence the Bay Area's discussions about and activities related to science, the environment, and nature, with a particularly local focus. Rockman et al (REA), a San Francisco-based research and evaluation organization, conducted an evaluation of QUEST programming and activities over the course of several years. The evaluation examined general QUEST audiences, formal and informal educators' use of QUEST, and KQED's development and maintenance of a partnership among a number of Bay Area science and environmental organizations. The results of REA's evaluation activities are presented here in a series of five reports for general audiences, educators, and partners. Overall Evaluation Over the course of three years, this evaluation discovered that KQED QUEST successfully serves a wide variety of audiences with different backgrounds, needs and interests. Across these diverse groups, a number of common interests emerged, such as: environmental issues, local concerns and organizations, high quality content, and current science information. During the grant period, the QUEST Interactive team expanded its web presence, offering the expanding online audiences new ways to interact with the content while simultaneously producing content for radio and television broadcast. The Education staff spread both QUEST content and digital media technology skills to science educators and students throughout the Bay Area. Throughout the process, the QUEST team has taken information gleaned from evaluation and implemented changes and enhancements - from mid-session tweaks to educator workshops to the adjustment of partner relationships and participation. General Audiences In Year 1, REA administered two audience surveys to gather baseline, current participation, and media use information from KQED audience members. To further understand audience members' engagement with and interest in science, environmental, and nature activities over time, REA recruited a subset of respondents from the Year 1 surveys to participate in a longitudinal audience panel over the course of six months in Year 2. In Year 3, the evaluation shifted to the growing number of audience members accessing and using QUEST's online content and resources. The data from the KQED QUEST Online Audience Survey found in the Online Audience Report complements the picture of KQED audiences' interests in and interactions with science gathered in prior years of the evaluation. Who are Online Audiences? The majority of Online Audience members (69%) live in the Bay Area. They represent a broad range of ages (50-50 split, 18-44 and 45-65+) and a good gender mix (M=59%, W=41%). These online audience members tend to be well educated; thirty percent have a Master's or professional degree. How do Online Audiences interact with new media? These online audience members differ quite a bit from the KQED general audience in their use of new media. While the majority of general audience members surveyed in Year 2 were not using new media, most of these Online Audience members reported using a wide variety of new media technologies and engaging in Web 2.0 activities. Moreover, their attitudes toward these technologies were quite positive. How do Online Audiences interact with QUEST online? Like the general KQED audience members, Online Audience members seek out QUEST content because they appreciate the uniqueness and quality of the content. They access this content online for convenience, and nearly one quarter visit the site a few times a week or more. When there, they are far more likely to seek out video pieces than any other content types. Educators The goals of the educator evaluation were twofold: to gather rich data regarding how educators implemented what they learned in a variety of KQED QUEST training workshops and outreach activities and to gather preliminary data on the impact QUEST educational activities have had on teaching and learning. To do so, REA employed a mixed-method evaluation approach that included observations of QUEST workshops, pre- and post-workshop educator surveys online surveys, and a series of in-depth case studies. Year 3 specifically focused on (a) an understanding of educators accessing QUEST's online content and resources, (b) the 2008-2009 QUEST Science Education Institute, and (c) real-world applications of QUEST content and digital media technologies. This summation report includes separate reports for each of these areas. Who are Online Educators? More than half of Online Educators teach high school students (59.5%) in public schools (56.8%), and between 40 and 50% teach science courses. Online Educators tend to be mid-career: 35.1% had been teaching for 4-9 years and 45.9% for 10 or more. Approximately three-quarters (73%) of the respondents are women (median age range 45-54), and over two-thirds (67.6%) live in Bay Area. A majority (53%) have attended some form of technology training/professional development, including 36% who attended a KQED training. How do Online Educators interact with new media? Like the general Online Audience members, these Educators use a wide variety of new media technologies and express extremely positive attitudes toward them. As Tables 1 and 2 below show, most Online Educators report accessing music/radio and TV online between a few times a month and a few times a week, 60% have created web pages, and nearly three-fourths have shared something they created with others online. How do Online Educators interact with QUEST? A number of Online Educators indicated that they regularly check the QUEST site for new content. Further, while one-fifth have not used QUEST with their students, more than 1/3 do so at least a few times a month. When Online Educators do use QUEST with students, they are most likely to use video segments (62.2%), followed by Educator Guides (45.9%). Online Educators find short, stand-alone video segments featuring cutting-edge content with associated educator guides the most useful. Science Education Institute Key Findings Program Educators felt that both their ability to use new media and knowledge of 21st Century skills increased as a result of their participation. Nearly all participants shared their newfound knowledge with their colleagues. As a result of participation, Program Educators reported increases in the use of a wide variety of digital media and QUEST resources with their students. Most participants felt that new media and QUEST content engaged their students and that using these resources added value to their classrooms.Partners KQED/QUEST staff initially focused on the development of the partnerships. Incorporating feedback from partner members gleaned through the evaluation, both KQED/QUEST staff and the partners themselves took a number of noteworthy steps to strengthen and reshape organizational relationships, processes, and collaborative program activities. As a result of focused, proactive outreach by QUEST staff, partners adopted a much more positive and collaborative project mindset. QUEST partner meetings became venues for learning new media skills and initiating new projects with QUEST and other partners. With a strong and working partnership model in place, the partner evaluation for Year 3 shifted to how member organizations were implementing the media and technology skills learned in partner workshops. Workshop participants gained both new skills and ideas for how to implement new media technologies on their organization's websites. Partners appreciated the sharing of ideas and the time for hands-on practice, which gave many the confidence to implement what they learned. Many organizations did add widgets and Google maps to their sites after the workshops. Ultimately, the partners looked forward to additional training sessions.

If you would like to edit a resource, please email us to submit your request.