Goodman Research Group, Inc. conducted a comprehensive multi-method external evaluation of the first season of the Design Squad TV series and outreach initiative. The broad evaluation goals were to: assess the extent to which children's knowledge, interest, and awareness of engineering increased as a result of watching the Design Squad series, document the implementation of community events resulting form the November 2006 Engineering Summit, and assess the effectiveness of the Afterschool Educators Guide with leaders and students.
DATE:
TEAM MEMBERS:
Peggy VaughanEmilee PressmanIrene GoodmanWGBH
Although there is a significant literature documenting the strength of informal science experiences in controlled or specialized settings, such as museums and science-focused youth programs it is not yet clear how best to regularly provide such opportunities in typical after-school programs--ones not established or specifically supported to test or implement grant-funded curricula or STEM approaches under specialized conditions. What's more, the majority of the 6.5 million children enrolled in after-school programs participate in typical programs. This study addressed this gap in the
Listening to the River (LTTR) is a watershed science education project funded by the National Science Foundation. Its aims are to deliver education experiences in the local area, Traverse City, Michigan, and also to develop a model that could be replicated in other locations. Inverness Research was contracted by the Listening to the River project to conduct both formative and summative evaluations. Our work began in 2005 when the project received a planning grant, and continued through the life of the project. Primarily through interviews and product reviews, along with some direct program
DATE:
TEAM MEMBERS:
Mark St. JohnHeather MitchellDawn RoblesElizabeth HorschLaura StokesLand Information Access Assocation
The Learning Places evaluation went beyond the traditional approach of determining how well the project met its goals. In addition to that approach, we identified grounded theory. Five areas of theory and related hypotheses emerged: STEM learning, agents of change, teens as designers, community partner engagement and national collaboration. Each is described in detail in the full report, along with findings related to each of the project goals.
Veridian inSight performed an evaluation study in fall of 2009 to evaluate the effectiveness of an online training developed as an outreach component of the FETCH! with Ruff Ruffman series (http://pbskids.org/fetch/). The FETCH! Hands-On Science Training was designed for anyone who wants to lead science activities with elementary-age kids (including after-school providers, teachers, camp counselors, librarians, museum staff, parents, and others). The goals of the evaluation study were to assess the extent to which the FETCH! training was successful at:· Preparing afterschool educators
In 2006, the Institute for Advanced Learning and Research (IALR) received a three-year grant from the National Science Foundation's Information/Innovative Technology Experiences for Students and Teachers (ITEST) division to create the Dan River Information Technology Academy (DRITA) for under-served high school students in rural Virginia. The only program of its kind in Southern Virginia, the program was designed to provide participating students with competencies in information technology (IT) and workforce skills. In addition, the program seeks to encourage students to graduate from high
DATE:
TEAM MEMBERS:
Irene GoodmanLorraine DeanMiriam KochmanHelena PylvainenColleen ManningKaren PetermanInstitute of Advanced Learning and Research
The evaluation was guided by four major questions: 1) How has math education changed at the afterschool programs sites over the course of the project? 2) Have children's experiences of informal math changed over the course of the project? 3) How have program staff changed in relation to math education? and 4) How is Mixing in Math being sustained at the program sites and disseminated to other programs and sites? Staff, program, and organizational factors all played a strong role in supporting high quality implementation of Mixing in Math. Mixing in Math was most successful where the
DATE:
TEAM MEMBERS:
Beth MillerTERC Inc.Kristin Lewis-Warner
FETCH with Ruff Ruffman is a daily half-hour PBS television series with accompanying Web and outreach activities targeted to 6- to 10-year olds. The program brings science learning to young children by uniquely blending live-action with animation, game show convention with reality programming, and humor with academics. The intended impacts are to 1) help the target audience develop interest, knowledge and skills necessary to do science; 2) train afterschool leaders to better facilitate science activities with kids; and 3) demonstrate how media can be used to teach substantive science and share the results of project evaluation with others in the field. The requested funds will allow the project to expand the science curriculum with 20 new half-hour episodes and expand the Web site, focusing on three new science themes that highlight topics of interest to this age group. The Web site will include four new science-based Web games that will allow kids to create and post content of their own design and contribute to nationwide data collection. A new FETCH Online Training resource will be created to help afterschool leaders to effectively engage in FETCH's hands-on science activities. American Institutes for Research (AIR) will conduct summative evaluation of the Online Training program.
This Pathways Project connects rural, underserved youth and families in Eastern Washington and Northern Idaho to STEM concepts important in sustainable building design. The project is a collaboration of the Palouse Discovery Science Center (Pullman, WA), Washington State University and University of Idaho, working in partnership with rural community organizations and businesses. The deliverables include: 1) interactive exhibit prototype activities, 2) a team cooperative learning problem-solving challenge, and (3) take-home materials to encourage participants to use what they have learned to investigate ways to make their homes more energy-efficient and sustainable. The project introduces youth and families to the traditionally difficult physics concept of thermal energy, particularly as it relates to sustainable building design. Participants explore how building materials and their properties can be used to control all three types of heat transfer: conduction, convection, and radiation. The interactive exhibit prototypes are coupled with an Energy Efficient Engineering Challenge in which participants, working in cooperative learning teams, use information learned from the exhibit prototype activities to retrofit a model house, improving its energy efficiency. The project components are piloted at the Palouse Discovery Science Center, and then travel to three underserved rural/tribal communities in Northern Idaho and Eastern Washington. Front-end and formative evaluation studies will demonstrate whether this model advances participant understanding of and interest in STEM topics and careers. The project will yield information about ways that other ISE practitioners can effectively incorporate cooperative learning strategies in informal settings to improve the transferability of knowledge gained from exhibits to real-world problem-solving challenges, especially for rural and underserved audiences. This project will also provide the ISE field with: 1) a model for increasing the capacity of small, rural science centers to form collaborative regional networks that draw on previously unused resources in their communities and provide more effective outreach to the underrepresented populations they serve, and 2) a model for coupling cooperative learning with outreach exhibits, providing richer experiences of active engagement.
This paper was prepared for the symposium, "Learning Science in Out-of-School Time: Research Directions," presented at the 2011 annual meeting of the American Educational Research Association, New Orleans. The authors discuss research efforts related to the nature of afterschool science offerings in publicly funded afterschool programs for elementary school students in California and the sources of support for science programming and afterschool staff development. Findings from the Afterschool Science Network (ASN) are outlined.
DATE:
TEAM MEMBERS:
Barbara MeansAnn HouseCarlin Llorente
During the spring of 2006, American Institutes for Research (AIR) conducted an evaluation study on behalf of WGBH. The purpose of the study was to gather data related to the effectiveness of the FETCH! Activity Guide, which was designed to extend the teachings of a new children's show, "FETCH! with Ruff Ruffman. The four main study objectives were to: Assess the activities' appeal for children (for example, do children enjoy the activities, do they realize they are learning about science, etc.?) Assess whether the facilitators liked the Activity Guide and deemed it appropriate for their after
Explore It! Science Investigations in Out-of-School Programs, funded by the National Science Foundation (NSF), was a collaboration among the Center for Science Education at the Education Development Center (EDC), six science and children museums, after-school centers across the United States, and the National Institute for Out-of-School Time (NIOST). The project primary goals were to develop and implement hands-on, inquiry-based units of activities for out-of-school programs for children ages 8-12 and to develop a support structure for after-school or out-of-school programs with science
DATE:
TEAM MEMBERS:
Patricia CampbellEducation Development CenterRosa Carson