Program evaluators from the Education Development Center (EDC) used a mixed-methods, quasi-experimental design to evaluate the impact on girls’ awareness and interest in science, technology, engineering, and mathematics (STEM). After the final year of the project, EDC delivered a summative report to Techbridge Girls (TBG), which was based on data collected during the five-year grant period, with a particular focus on the final year that grant funds supported programming (2017-18). Data included pre- and post-surveys with TBG participants and comparison students, participant focus groups, and
Children Investigating Science with Parents and Afterschool (CHISPA) was a collaboration between the Phillip and Patricia Frost Museum of Science, UnidosUS (formerly National Council of La Raza), and the ASPIRA Association that took place from 2014-18. CHISPA sought to address the disparity in science achievement among Latino and non-Latino children through local-level partnerships between science museums in metropolitan areas with growing Latino populations and UnidosUS and ASPIRA affiliate organizations serving the same communities through afterschool programs.
Partners included the
The starting point of the Evaluating Evaluation project was our impression that despite the substantial resources that are spent on the summative evaluation of museums and galleries the research has little impact and largely remains ineffectual.
With support from the Wellcome Trust and the Heritage Lottery Fund, we set out to see why this seems to be the case and to explore whether there are ways in which the impact of summative evaluations on knowledge and practice might be enhanced. To this end, we reviewed a substantial number of reports, undertook a range of interviews and held two
The most important consideration in evaluating chemistry outreach efforts is how to best use the evaluation to serve project needs. Evaluation should be about making programs more effective—at communicating ideas, changing attitudes, inspiring action, or reaching wider audiences, for example. A well-conducted evaluation typically contributes to the quality of a project by helping its leaders better define their goals, identify important milestones and indicators of success, and use evidence to support ongoing improvements. At its best, evaluation is an integral part of project design and
This evaluator reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It reflects on the nature of learning and challenges assumptions of outcomes from engaging in informal learning experiences.
This leadership reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It provides an organizational leadership perspective on evaluation.
This practitioner reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It discusses improving practice through reflection.
In late 2012, Providence Children’s Museum began a major three-year research project in collaboration with The Causality and Mind Lab at Brown University, funded by a grant from the National Science Foundation (1223777). Researchers at Brown examined how children develop scientific thinking skills and understand their own learning processes. The Museum examined what caregivers and informal educators understand about learning through play in its exhibits and how to support children’s metacognition – the ability to notice and reflect on their own thinking – and adults’ awareness and appreciation of kids’ thinking and learning through play. Drawing from fields like developmental psychology, informal education and museum visitor studies, the Museum’s exhibits team looked for indicators of children’s learning through play and interviewed parents and caregivers about what they noticed children doing in the exhibits, asking them to reflect on their children’s thinking. Based on the findings, the research team developed and tested new tools and activities to encourage caregivers to notice and appreciate the learning that takes place through play.
The Wildlife Conservation Society and Good Shepherd Services (a youth development and education agency) are implementing and evaluating a school-to-career model program that consists of afterschool and weekend programming for high school students at four New York City area zoos and an aquarium, followed by post-participation tracking, support, and mentoring. The goal is to promote affective, cognitive and behavioral outcomes among 150 low-income minority youth necessary to pursue careers in the wildlife sciences.
The Bridging the Gap project is (1) developing a science career program that includes hands-on, technology-enriched, science learning experiences at zoos/aquaria; career building services, mentoring, and long-term tracking and support, (2) forming a community of minority students who have the knowledge and skills to pursue wildlife careers, (3) generating research findings on the short-term and long-term effectiveness of the program, and (4) disseminating information about the project's resources and findings to other informal science education institutions around the nation for replication. The evaluation plan measures a variety of outputs, outcomes and impacts that include short-term and long-term cognitive and affective variables. Data collection methods include student activity monitoring and pre-post testing.
The project addresses a compelling personnel issue documented by the American Zoo and Aquarium Association - the small number of minority science professionals working in zoos and aquariums. Because few programs currently exist to help minority students enter the wildlife science professions, this project fills an important programmatic need and serves as a model workforce program that can be replicated by other informal science education organizations around the country. The project's key strategic impact is its capacity to broaden participation in the wildlife sciences by expanding the science professional pipeline beginning in high school.
DATE:
-
TEAM MEMBERS:
Karen TingleyChanda BennettDon LisowyBrian JohnsonEmily StoethCourtney Wiggins
This project will be conducted by a team of investigators from North Carolina State University. The principal investigator proposes to examine the characteristics, motivations, in and out-of-school experiences, informal science activities, and career trajectories of 1000 science hobbyists and "master hobbyists." Master hobbyists are individuals who have developed science expertise and spend considerable free time engaging in science as a leisure activity. Master science hobbyists are found across most areas of science (e.g. birdwatchers, amateur astronomers). This research will determine who these individuals are, their career pathways, how they engage in science activities and what motivates, sustains, and defines their science interests. One of the particular goals of this research is to develop new understandings of how science hobby interests develop for women and underserved minorities. In the proposed research investigators will use the results of interviews and surveys to identify contextual factors that influence the motivational processes that, in turn, influenced choices of careers and contribute to ongoing choices in hobby and citizen science activities. Of interest in this study is how citizen scientists who are also serious hobbyists differ from master science hobbyists. Research on citizen scientists has shown that this group is highly motivated by collective motives (such as a desire to help others and further science), whereas this may not be the case with the master science hobbyist. Two groups will be sampled: a) birdwatchers and b) amateur astronomers. This sampling model will allow investigators to contrast their findings by: 1) those who have selected a science career versus those that did not select a science career, 2) those who participate in citizen science activities and those that do not, and 3) those who are birdwatchers (greater mathematical components) and those who are amateur astronomers (lesser mathematical components). Additional coding and analyses will examine any differences in the evolution of bird watching and astronomy hobbies. The results of this research will be examined in light of existing motivational and sociocultural models of career selection. This research will document differences in the perceived motivational elements that influenced master science hobbyists/citizen scientists to choose a science career or not. The results can inform federal, state, and local policies for supporting youth and adults engaged in free choice learning. Results of this research will inform the design of intervention/recruitment programs and ISE outreach initiatives. Potential audiences include ISE institutions (e.g. museums and science centers), organizations with links to STEM (e.g. scouts, boys/girls clubs) and pre- and college initiatives that seek to influence career choices and life-long science interests. The proposed cross-disciplinary approach will promote new understandings of complex issues related to motivation, retention, career selection, leisure activities, engagement with formal and informal educational environments, gender and ethnicity, communities of practice and changes in interests over time. Members of the advisory board have expertise in assessment and measurement and will work closely with the project team to conduct a detailed examination of methodologies and analyses at all phases of the project.
The Adler Planetarium, Johns Hopkins University, and Southern Illinois University-Edwardsville are investigating the potential of online citizen science projects to broaden the pool of volunteers who participate in analysis and investigation of digital data and to deepen volunteers' engagement in scientific inquiry. The Investigating Audience Engagement with Citizen Science project is administering surveys and conducting case studies to identify factors that lead volunteers to engage in the astronomy-focused Galaxy Zoo project and its Zooniverse extensions. The project is (1) identifying volunteers' motivations for joining and staying involved, (2) determining factors that influence volunteers' movement from lower to higher levels of involvement, and (3) designing features that influence volunteer involvement. The project's research findings will help informal science educators and scientists refine existing citizen science programs and develop new ones that maximize volunteer engagement, improve the user experience, and build a more scientifically literate public.
DATE:
-
TEAM MEMBERS:
Karen CarneyMichael RaddickPamela Gay