Dot Diva pilot test results

In 2008, the WGBH Educational Foundation, along with the Association of Computing Machinery, was awarded a grant from the National Science Foundation, Directorate for Computer and Information Science and Engineering, under the Broadening Participation in Computing Program (NSF 0753686). The purpose of the grant was to develop a major new initiative to reshape the image of computing among college-bound high school students. Based on its market research results, WGBH developed a website and other resources that were intended for use by teachers, parents and students. Concord Evaluation Group (CEG) was hired by WGBH to conduct a pilot test of the resources and materials. The pilot test was performed in an urban high school setting in Massachusetts during the fall semester of the 2010-11 academic year. We recruited a sample of teachers and students from the high school to participate in the pilot test. The pilot test sample did not include the entire student body. Rather, because the Dot Diva resources were intended to target college-bound high school girls, the pilot test included teachers and female students from advanced math classes only. Students from all four grades (9th - 12th) were eligible. Four math and science teachers were chosen to participate (and expressed interest) in the pilot test. At the start of the study, teachers completed an online, pre-test survey to measure their: Ability to help girls prepare for computing careers (self-efficacy). Attitudes towards computing careers. Perception of how interested their female students were in computing careers. After the pre-tests were completed, WGBH provided the four teachers with resources to use during the pilot test. Teachers were encouraged to distribute brochures to all of their college-bound, female students. Teachers also placed posters in their classrooms and in the school hallways. In addition, three guest speakers (with professional backgrounds in computer science) were invited to the school to talk with college-bound students about their careers. 3 We also asked teachers to allow students to visit the website during their class time or to remind students that they should try to visit the website at home (the website address was also included in the brochures and on the posters). At the end of the semester, CEG administered Web-based post-test surveys to the teachers, parents, and students who completed pre-test surveys. This report describes the results of the pilot test and offers recommendations for additional field testing and future implementation of the initiative. In addition, we reviewed comments posted on the Dot Diva website and the Dot Diva Facebook page to gather additional data on what users liked and didn't like about the Dot Diva Web-based resources. We would recommend that WGBH consider dropping from Dot Diva those resources that either didn't get used or were reported to have limited usefulness in the pilot test. This would include the brochures and the posters. We also recommend that WGBH try to identify ways to reach parents other than through their daughters or their daughters' schools. With respect to the Dot Diva program, as a whole, we recommend that WGBH consider: (1) developing a program that could be used in after-school programs and provide a more intensive focus on the messages contained in the initiative, or (2) developing a program that would more easily plug into existing high school math or science curricula so that teachers can better integrate the resources into their daily lessons, and/or (3) including hands-on activities that teachers could do with students in the classroom. Doing so should enable teachers to offer students a richer experience with a deeper exploration of the different fields associated with computing and a better explanation of how to prepare for such careers. With respect to the website and the webisode, we recommend the following: Provide more detail about computer-related jobs and the responsibilities expected of individuals in those professions. Students, parents, teachers, and online users all commented that there were insufficient technical details provided on the website. Related to the previous point, we recommend that WGBH include a lot more discussion of, and demonstrations of, technical issues (e.g., code and how code is used in different career settings) in future webisodes. Some students failed to even see a connection between the webisode and computing, and some webisode viewers commented that the lack of inclusion of technical information made the webisode uninformative. Some website users and some pilot test participants reported that they found the website and webisode to be too girly. Others were highly offended at the way some of the women were portrayed and the lack of substance in the discussions presented. We would recommend that WGBH pilot test each webisode with both female students and female computing professionals to ensure that the intended message is not unintentionally overshadowed with mixed messages. While the humor was appreciated by many viewers, some took offense to the use of humor at other people's expense or at humor that actually seemed to reinforce stereotypes about female computer professionals (e.g., the woman who was confused with a man). We recommend that WGBH consider, in future webisodes, how humor might be used differently (because it is important) and in a way that relies less on taking jabs at others. Finally, to ensure widespread use of the resources, we recommend that WGBH offer the website in different languages and provide closed captioning for all videos. We should note that no one in the sample reported that the website was inaccessible rather these were general suggestions offered by website users.

If you would like to edit a resource, please email us to submit your request.