The broad purpose of this project is to contribute to capacity-building efforts—to strengthen the evaluation of programs and learning experiences in informal science education (ISE). We focus on improving the quality of summative evaluations, which have been called upon to inform decision-making and practice, contribute knowledge to the field, and help make the case for the value of informal learning. We previously developed a framework that synthesizes key elements of a high-quality summative evaluation. The framework has three dimensions: (a) examine the underlying rationale of the program, exhibition, experience, or intervention being evaluated; (b) balance methodological rigor with sensitivity to the informal context; and (c) prioritize uses of the evaluation by addressing stakeholders’ needs. Evaluators may draw on all three dimensions to provide summary judgments on the value of what was evaluated. We extend our work in two ways: (1) Examine how the framework could be used as a guiding lens for planning or conducting future or current summative evaluations. We will work with partners in informal science institutions to support and document their evaluation activities along the dimensions of evaluation quality, with the intent of improving the framework and creating concrete exemplars of its application in practice. (2) Conceptualize alternative models for enhancing professional training in ISE evaluation. We will research current programs and practices for building technical evaluation capacity, looking for promising and innovative approaches that include apprenticeship or “hands-on” experiences. This 18-month project (January 2015 through June 2016) is funded by the Gordon and Betty Moore Foundation.
Goodman Research Group, Inc., (GRG), Cambridge, MA, conducted the formative evaluation of The Music Instinct project. The NSF-funded project aims to bring to PBS viewers the strong evidence of the connections between music and science, as well as to facilitate a deeper understanding of both fields. The Music Instinct project, presented by WNET/Thirteen, in collaboration with Mannes Productions, includes a two-hour television program, a website, and ancillary educational materials. The purpose of the formative evaluation is to obtain timely information to support and guide the producers as they
DATE:
TEAM MEMBERS:
Rucha LondheMiriam KochmanNivedita RanadeIrene F GoodmanWNET/ThirteenMannes Productions Inc.
With the Role of Media in Supporting Informal Science Learning project, the Institute for Learning Innovation (ILI) and Grunwald Associates sought to establish a “national learning initiative” to explore the intersection of media and informal science learning. To do so, ILI proposed an initial conference followed by the development of a website and online community. The National Science Foundation funded this project, with additional funding provided by the National Parks Service. Held in March 2009, the 1.5 day conference was designed to be a “first step” in the development of a conversation
DATE:
TEAM MEMBERS:
Elizabeth Bandy, Ph.D.Institute for Learning Innovation
The objective of this project is to develop resources for STEM learning by redesigning and expanding the "Jonathan Bird's Blue World" website; adding components to enable teachers and students to search episodes for specific themes, locations, or scientific concepts; and enhancing the lesson plans to explicitly match the content standards for teaching science. One of the major objectives is to make the "Jonathan Bird's Blue World" web site content more widely accessible as an open source via an internet connection with a dynamic search capability. The project adds a significant amount of educational material and uses ocean topics to illustrate nationally accepted standards for science learning. It is applicable to both formal and informal environments and contains a multi-faceted approach to achieve its goal including the following objectives: (a) build a web site search capability that includes National Science Educational Standards (NSES) and Ocean Literacy Principles with a link to segments illustrating specific NSES; (b) create webisodes that feature specific NSES and develop an interactive map on the web site; (c) create a user survey tool and outreach for these improvements. By partnering with researchers and educators to incorporate nationally accepted standards for science learning into the website redesign, the project has a broad impact on professional development for teachers. It has the potential to reach diverse audiences in schools, homes and afterschool programs. "Jonathan Bird's Blue World" is designed to complement other educational materials and provide a useful resource for teachers everywhere, improving public scientific literacy in informal and formal settings.
DATE:
-
TEAM MEMBERS:
Christine BirdBeth MeInikasJonathan Bird
The goal of this engineering education project entitled EXTRAORDINARY WOMEN ENGINEERS (EWE) is to encourage more academically prepared high school girls to consider engineering as an attractive option for post-secondary education and subsequent careers in order to increase the number of women who make up the engineering workforce. Specific project objectives are to: 1) mobilize America's more than one million engineers to reach out to educators, school counselors, and high school girls with tested messages tailored to encourage participation in engineering education and careers; 2) help high school counselors and science, math, and technology teachers to better understand the nature of engineering, the academic background needed to pursue engineering, and the career paths available in engineering; 3) equip high school counselors and teachers to share this information with students, especially girls; and 4) reach out to girls directly with messages that accurately reflect the field of engineering and will inspire girls to choose engineering. The WGBH Educational Foundation has partnered with the American Association of Engineering Societies (AAES), American Society of Civil Engineers (ASCE), and a coalition of more than 50 of the country's engineering associations, colleges, and universities to fundamentally shift the way the engineering and educational communities portray engineering. Based on a needs assessment performed in 2004, the EWE coalition embraces a communication strategy that focuses on the societal value and rewards of being an engineer, as opposed to the traditional emphasis on the process and challenges of becoming an engineer. This project represents a nationwide outreach effort that includes training opportunities for engineers; targeted Web-based and print resources for students, school counselors and teachers, and engineers; and a range of outreach and marketing activities.
DATE:
-
TEAM MEMBERS:
Julie BenyoPatrick NataleF. Suzanne Jenniches
This project will produce a multi-part radio project including eight half-hour documentaries, 40-50 short radio features, an audio clearinghouse and a website on scientific research in the Polar Regions. The content of the programs support the goals of IPY. The project will be produced with four international radio partners: the science units of The Australian Broadcasting Co., the BBC World Service, Radio Deustche-Welle and Radio New Zealand. These international collaborators will look at issues such as the influence of conditions in Polar Regions on global climatic change, how animals adapt to rapid environmental change, survival in extreme environments and processes of change among native people in the Polar regions. The programs will reach a large audience in the United States as well as internationally through the collaborating partners. The clearinghouse/website will be designed to provide organized learning resources and an audio archive of the project's radio programs and archival interviews and sounds for use by both the general public and professional audiences. There will be both formative and summative evaluation of the programs and website.
This collaborative project aims to establish a national computational resource to move the research community much closer to the realization of the goal of the Tree of Life initiative, namely, to reconstruct the evolutionary history of all organisms. This goal is the computational Grand Challenge of evolutionary biology. Current methods are limited to problems several orders of magnitude smaller, and they fail to provide sufficient accuracy at the high end of their range. The planned resource will be designed as an incubator to promote the development of new ideas for this enormously challenging computational task; it will create a forum for experimentalists, computational biologists, and computer scientists to share data, compare methods, and analyze results, thereby speeding up tool development while also sustaining current biological research projects. The resource will be composed of a large computational platform, a collection of interoperable high-performance software for phylogenetic analysis, and a large database of datasets, both real and simulated, and their analyses; it will be accessible through any Web browser by developers, researchers, and educators. The software, freely available in source form, will be usable on scales varying from laptops to high-performance, Grid-enabled, compute engines such as this project's platform, and will be packaged to be compatible with current popular tools. In order to build this resource, this collaborative project will support research programs in phyloinformatics (databases to store multilevel data with detailed annotations and to support complex, tree-oriented queries), in optimization algorithms, Bayesian inference, and symbolic manipulation for phylogeny reconstruction, and in simulation of branching evolution at the genomic level, all within the context of a virtual collaborative center. Biology, and phylogeny in particular, have been almost completely redefined by modern information technology, both in terms of data acquisition and in terms of analysis. Phylogeneticists have formulated specific models and questions that can now be addressed using recent advances in database technology and optimization algorithms. The time is thus exactly right for a close collaboration of biologists and computer scientists to address the IT issues in phylogenetics, many of which call for novel approaches, due to a combination of combinatorial difficulty and overall scale. The project research team includes computer scientists working in databases, algorithm design, algorithm engineering, and high-performance computing, evolutionary biologists and systematists, bioinformaticians, and biostatisticians, with a history of successful collaboration and a record of fundamental contributions, to provide the required breadth and depth. This project will bring together researchers from many areas and foster new types of collaborations and new styles of research in computational biology; moreover, the interaction of algorithms, databases, modeling, and biology will give new impetus and new directions in each area. It will help create the computational infrastructure that the research community will use over the next decades, as more whole genomes are sequenced and enough data are collected to attempt the inference of the Tree of Life. The project will help evolutionary biologists understand the mechanisms of evolution, the relationships among evolution, structure, and function of biomolecules, and a host of other research problems in biology, eventually leading to major progress in ecology, pharmaceutics, forensics, and security. The project will publicize evolution, genomics, and bioinformatics through informal education programs at museum partners of the collaborating institutions. It also will motivate high-school students and college undergraduates to pursue careers in bioinformatics. The project provides an extraordinary opportunity to train students, both undergraduate and graduate, as well as postdoctoral researchers, in one of the most exciting interdisciplinary areas in science. The collaborating institutions serve a large number of underrepresented groups and are committed to increasing their participation in research.
DATE:
-
TEAM MEMBERS:
Tandy WarnowDavid HillisLauren MeyersDaniel MirankerWarren Hunt, Jr.
The institution is The Ohio State University at Lima, the university partners are the University of North Carolina at Greensboro and Fayetteville State University. It's About Discovery is a unique partnership to engage students and teachers in critical thinking skills in STEM content areas. The Ford Partnership for Advanced Studies (PAS) new science curriculum is the foundation for the project which will include over 700 students and 20-25 teachers. While the primary focus is on students, throughout the life of the project all teachers will participate in professional development focusing on the PAS units to ensure the quality teaching and understanding of the content. Technology will be integrated throughout the program to enable students to create inquiry based projects across state lines and for teachers to continue their professional development opportunities. Community partners will serve as mentors, host field trips, and engage in on-line conversations with students. An interactive website will be created for both teachers and students. The focus is on 8th grade science as it relates to STEM careers, 9th grade physical science and 10th science and mathematics. We are implementing a new Ford PAS curriculum module, Working Towards Sustainability, which comprises of four modules: We All Run on Energy, Energy from the Sun, Is Hydrogen a Solution? and The Nuclear Revolution. Teachers across states will engage in a new professional development model. Students will create projects through on-line conversations. A website will be created for project participants and the ITEST community. These hands-on, inquiry-based learning experiences engage students and prepare and encourage them to pursue science, engineering, and technology in high school and beyond. All PAS curricula use real world experiences, open-ended problems and result in real world applications. Assessments are on-going and inquiry driven. Teamwork and on-line resources and research are built into the curriculum design. The evaluation consists of a multi-method pre-post design. Teachers complete a Pre Survey at the beginning of the program and then again at the end of the school year. Students complete a Pre Survey at the beginning of the school year and a post survey at the end of the school year. In addition, teachers share students' scores on curriculum assessments completed throughout the year, including student scores on the Comprehensive Adult Student Assessment System's (CASAS) Assessment of Critical Thinking in Science writing tasks.
Concord Evaluation Group (CEG) performed an evaluation study in the fall of 2010 on behalf of WGBH to evaluate the effectiveness of an online, interactive webisode developed as an educational component of the FETCH! with Ruff Ruffman series (http://pbskids.org/fetch/). The webisode was not connected to the FETCH! website at the time of the evaluation study, but will be after the study is complete. With an emphasis on building math skills, the interactive webisode was designed to teach and engage elementary-aged kids (ages 6 – 10) to identify and create combinations. In addition to identifying
SciGirls is a national dissemination project, which puts resources from the PBS science series DragonflyTV into the hands of outreach professionals at PBS stations and educators in after school programs for girls. The current project leverages PBS' nationwide network of member stations to connect the educational research community with practitioners in the field. Thus far SciGirls has trained over 100 educators and community leaders and reached 2,300 girls in grades 3 through 8. SCIGIRLS MUSEUM ADVENTURES has four objectives: 1) to provide museum educators with DragonflyTV videos that model authentic inquiry in museum settings; 2) to expand SciGirls activity guides with new museum-based activities and research-based strategies specifically for museum educators; 3) to create a set of online, streaming videos that demonstrate best practices in gender-inclusive teaching; and 4) to facilitate feedback between our participants and the research community and deepen our understanding of the most effective ways to engage girls in STEM activities. Intellectual Merit--The strength of SciGirls lies in its comprehensive multimedia approach and its foundation in the inquiry-based strategies defined in the National Science Education Standards. The videos provided in SciGirls emphasize the process of science, rather than a collection of science facts. They provide real-world models of inquiry that all girls can do. Taken together, the SciGirls resources stimulate discussion, build confidence and pave the way for girls to investigate science questions on their own. The educational strategies provided by SciGirls are based in research into gender- inclusive STEM teaching and learning, translated into strategies that can be easily used by after school educators to create successful STEM experiences for girls. Broader Impact--SCIGIRLS MUSEUM ADVENTURES will provide museum educators at ten sites with materials that can be used in their programs for years to come. The entire set of resources--streaming videos and Activity Guides--will be available on DragonflyTV's Web site at www.pbs.org. The outcomes of the project will be shared with the informal science education research community. Findings will be reported at the annual PBS National Center for Outreach Conference
Produced by Thirteen/WNET New York and Nelvana International, the Award-winning series CYBERCHASE is the only mathematics series for children on American television. Designed for kids ages 8 to 12 and packed with mystery, humor, and action, each episode delivers positive messages about math by teaching concepts in a fun way that kids can understand. The goal of this formative evaluation for Season 3 evaluation was two-fold: 1) to observe and assess the performance of third-graders on tasks related to two math topics in development for Season 3 (Angle & Distance and Logic & Reasoning) and 2) to
This formative evaluation gathered feedback from parents and their fourth grade children in response to two activities included in the Cyberchase at Home outreach materials. The user-based feedback assisted with the design of new outreach materials. The general goals for the research were to explore reactions to the activity card format; assess appeal and difficulties in implementation of two activities; estimate comprehension of activity content; and evaluate parental interest in further activities and workshops. Cyberchase is the Emmy Award-winning mathematics series and website on PBS KIDS