This is the summative evaluation of Statistics for Action (SfA). The mixed-methods evaluation included both process (project implementation) and impact (project effectiveness) assessments. It was posited a cascade-like impact of SfA, in which new materials would be developed by TERC staff; a host of environmental organizations would be trained to utilize them with grass roots community groups; and these groups would then incorporate SfA into their ongoing environmental campaigns. Ultimately, it was theorized, the public messaging around environmental issues would be strengthened by SfA’s
DATE:
TEAM MEMBERS:
Arbor Consulting PartnersMargaret ConnorsMindy FriedMadeleine Taylor
The article describes the purpose of the evaluation for Statistics for Action (SfA): to learn to what extent SfA's set of materials and resources and training developed by math educators with input from environmental organizers, could increase numeracy among environmental organizers and the community members they serve. For this generally unpopular content (math) in an unusual context (environmental campaigns), the author describes project and evaluation design choices that worked, and those that didn’t.
This paper shares research findings that explore the link between conceptual understanding, free exploration of hands-on science center exhibits, and Piagetian cognitive developmental levels. Implications on museum exhibits and field trip experiences are examined as well.
In this paper, Alan J. Friedman, director of the New York Hall of Science, discusses the importance of evaluation, specifically formative evaluation, for measuring success at a museum and fulfilling mission statements. Friedman presents four real examples of exhibit projects, in which the directors or exhibit directors involved elected to expend funds for formative evaluation. Friedman goes on to discuss the benefits of this research and what surprising information staff came to know as a result.
In this article, Jeff Hayward of People, Places & Design Research outlines the evaluation process by summarizing the evaluation of "The Estimating Game," a traveling exhibit developed by the Children's Museum of Boston. The study assessed the effectiveness of the exhibit as installed at the Children's Museum, the effectiveness of "mock-ups" of parts of the exhibit as they were developed, and the effectiveness of teaching children the concept of estimating.
This article discusses a 1988-1990 study that analyzed the effectiveness of a collaborative effort between a museum and a school system to build an integrated curriculum package. The partners included the York County School System (VA) and the Yorktown Victory Center (operated by the Jamestown-Yorktown Foundation). The theme of the curriculum was 18th Century Medicine and the unit was designed to enhance the science, math, and social studies instruction of fourth graders.
DATE:
TEAM MEMBERS:
Ronald GieseJudy Davis-DorseyJoseph Gutierrez
Project LIFTOFF works with local, regional, and national partners to engineer statewide systems for Informal Science Education that inspire: YOUTH to pursue STEM education and careers through increased opportunities for quality, hands-on STEM learning. AFTERSCHOOL STAFF to facilitate STEM learning experiences that contribute to the overall STEM education and aspirations of youth in their programs. PROGRAM ADMINISTRATORS to encourage and support staff in the integration of STEM enrichment into the daily programming. STATE LEADERS to sustain and expand afterschool learning opportunities so that all students have access to engaging STEM experiences outside of the regular school day. Project LIFTOFF is dedicated to the development of the following essential elements of statewide systems for informal science education:
Access to appropriate STEM Curriculum for youth of all ages, abilities, and socio-cultural backgrounds that meets the needs and interests of individual community programs
Systematic STEM Professional Development that matches individual skills in positive youth development with abilities to facilitate discovery and science learning
A diverse Cadres of Trainers who will deliver the professional development, technical assistance and curriculum dissemination in their local communities
Authentic Evaluation of informal science efforts that determine the impacts on youth aspirations and the capacity of youth programs to provide quality STEM experiences
Local STEM education leadership to identify the ways in which collaborative education efforts can advance the development of 21st Century Skills and the preparedness for STEM workforce and higher education
Partnerships in support of youth development and informal science education that convene local, regional, and statewide organizations and stakeholders
To advance national initiatives and states' sySTEM engineering efforts, LIFTOFF coordinates an annual convening, the Midwest Afterschool Science Academy, that brings together national informal science experts, system leaders and youth development professionals to elevate the levels of science after school. The 5th MASA will be in the spring of 2014 in Kansas City, MO
DATE:
TEAM MEMBERS:
Missouri AfterSchool NetworkJeff Buehler
The Learning and Youth Research and Evaluation Center (LYREC) is a collaboration of the Exploratorium, Harvard University, Kings College London, SRI International and UC Santa Cruz. LYREC provides technical assistance to NSF AYS projects, collects and synthesizes their impact data, and oversees dissemination of progress and results. This center builds on the Center for Informal Learning in Schools (CILS) that has developed a theoretical approach that takes into account the particular strengths and affordances of both Out of School Teaching (OST) and school environments. This foundation will permit strengthening the potential of the NSF AYS projects to develop strong local models that can generate valid and reliable data that can guide future investment, design and research aimed at creating coherence across OST and school settings. The overarching questions for the work are: 1. How can OST programs support K-8 engagement and learning in science, and in particular how can they contribute to student engagement with K-8 school science and beyond? 2. What is the range of science learning outcomes OST programs can promote, particularly when in collaboration with schools, IHE's, businesses, and other community partners? 3. How can classroom teachers and schools build on children's OST experiences to strengthen children's participation and achievement in K-12 school science Additionally, the data analysis will reveal: 1. How OST programs may be positioned to support, in particular, high-poverty, female and/or minority children traditionally excluded from STEM academic and career paths; and 2. The structural/organizational challenges and constraints that exist to complicate or confound efforts to provide OST experiences that support school science engagement, and conversely, the new possibilities which are created by collaboration across organizational fields. Data will be gathered from surveys, interviews, focus groups, evaluation reports, and classroom and school data.
Evaluators sought to determine impact on math attitude and content knowledge of students (3rd - 6th grade), and math literacy workers (high school and college students), based on pre-post testing and observation of YPP after school programs, in which college and high school students teach math games to elementary and middle school students in marginalized and vulnerable communities. The study focused on the Chicago YPP site, 1 of 7 in the YPP national network. Increases in math attitude scores were not statistically significant, however in some instances evaluators found significant increases
DATE:
TEAM MEMBERS:
Norman LedermanYoung People's Project, Inc.
Following a 2011 report by the National Research Council (NRC) on successful K-12 education in science, technology, engineering, and mathematics (STEM), Congress asked the National Science Foundation to identify methods for tracking progress toward the report's recommendations. In response, the NRC convened the Committee on an Evaluation Framework for Successful K-12 STEM Education to take on this assignment. The committee developed 14 indicators linked to the 2011 report's recommendations. By providing a focused set of key indicators related to students' access to quality learning, educator's
The intent of this project is to use social network methods to study networks of afterschool and informal science stakeholders. It would attempt to create knowledge that improves afterschool programs access to informal science learning materials. This is an applied research study that applies research methods to improving access to and enactment of informal science education programs across a range of settings. The investigators plan to collect data from 600 community- and afterschool programs in California, conduct case studies of 10 of these programs, and conduct surveys of supporting intermediary organizations. The analysis of the data will provide descriptions of the duration, intensity, and nature of the networks among afterschool programs and intermediary agencies, and the diffusion patterns of science learning materials in afterschool programs. The project will yield actionable knowledge that will be disseminated among afterschool programs, intermediary organizations, funding agencies, and policymakers to improve the dissemination and support of afterschool science learning opportunities. The project is focused on free-choice settings where every day the largest numbers of children attend afterschool programs at schools and in other community settings. It seeks information about what conditions are necessary for informal science programs to significantly impact the largest possible number of children in these settings.
DATE:
-
TEAM MEMBERS:
Barbara MeansAnn HouseCarlin LlorenteRaymond McGhee
Concord Evaluation Group (CEG) performed an evaluation study in the fall of 2010 on behalf of WGBH to evaluate the effectiveness of an online, interactive webisode developed as an educational component of the FETCH! with Ruff Ruffman series (http://pbskids.org/fetch/). The webisode was not connected to the FETCH! website at the time of the evaluation study, but will be after the study is complete. With an emphasis on building math skills, the interactive webisode was designed to teach and engage elementary-aged kids (ages 6 – 10) to identify and create combinations. In addition to identifying