Program evaluators from the Education Development Center (EDC) used a mixed-methods, quasi-experimental design to evaluate the impact on girls’ awareness and interest in science, technology, engineering, and mathematics (STEM). After the final year of the project, EDC delivered a summative report to Techbridge Girls (TBG), which was based on data collected during the five-year grant period, with a particular focus on the final year that grant funds supported programming (2017-18). Data included pre- and post-surveys with TBG participants and comparison students, participant focus groups, and
The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.
The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.
The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”
DATE:
-
TEAM MEMBERS:
Maritza MacdonaldMeryle WeinsteinRosamond KinzlerMordecai-Mark Mac LowEdmond MathezDavid Silvernail
This evaluator reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It reflects on the nature of learning and challenges assumptions of outcomes from engaging in informal learning experiences.
This leadership reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It provides an organizational leadership perspective on evaluation.
This practitioner reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It discusses improving practice through reflection.
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
'Be a Scientist!' is a full-scale development project that examines the impact of a scalable, STEM afterschool program which trains engineers to develop and teach inquiry-based Family Science Workshops (FSWs) in underserved communities. This project builds on three years of FSWs which demonstrate improvements in participants' science interest, knowledge, and self-efficacy and tests the model for scale, breadth, and depth. The project partners include the Viterbi School of Engineering at the University of Southern California, the Albert Nerken Engineering Department at the Cooper Union, the Los Angeles Museum of Natural History, and the New York Hall of Science. The content emphasis is physics and engineering and includes topics such as aerodynamics, animal locomotion, automotive engineering, biomechanics, computer architecture, optics, sensors, and transformers. The project targets underserved youth in grades 1-5 in Los Angeles and New York, their parents, and engineering professionals. The design is grounded in motivation theory and is intended to foster participants' intrinsic motivation and self-direction while the comprehensive design takes into account the cultural, social, and intellectual needs of diverse families. The science activities are provided in a series of Family Science Workshops which take place in afterschool programs in eight partner schools in Los Angeles and at the New York Hall of Science in New York City. The FSWs are taught by undergraduate and graduate engineering students with support from practicing engineers who serve as mentors. The primary project deliverable is a five-year longitudinal evaluation designed to assess (1) the impact of intensive training for engineering professionals who deliver family science activities in community settings and (2) families' interest in and understanding of science. Additional project deliverables include a 16-week training program for engineering professionals, 20 physics-based workshops and lesson plans, Family Science Workshops (40 in LA and 5 in NY), a Parent Leadership Program and social networking site, and 5 science training videos. This project will reach nearly one thousand students, parents, and student engineers. The multi-method evaluation will be conducted by the Center for Children and Technology at the Education Development Center. The evaluation questions are as follows: Are activities such as recruitment, training, and FSWs aligned with the project's goals? What is the impact on families' interest in and understanding of science? What is the impact on engineers' communication skills and perspectives about their work? Is the project scalable and able to produce effective technology tools and develop long-term partnerships with schools? Stage 1 begins with the creation of a logic model by stakeholders and the collection of baseline data on families' STEM experiences and knowledge. Stage 2 includes the collection of formative evaluation data over four years on recruitment, training, co-teaching by informal educators, curriculum development, FSWs, and Parent Leadership Program implementation. Finally, a summative evaluation addresses how well the project met the goals associated with improving families' understanding of science, family involvement, social networking, longitudinal impact, and scalability. A comprehensive dissemination plan extends the project's broader impacts in the museum, engineering, evaluation, and education professional communities through publications, conference presentations, as well as web 2.0 tools such as blogs, YouTube, an online social networking forum for parents, and websites. 'Be a Scientist!' advances the field through the development and evaluation of a model for sustained STEM learning experiences that helps informal science education organizations broaden participation, foster collaborations between universities and informal science education organizations, increase STEM-based social capital in underserved communities, identify factors that develop sustained interest in STEM, and empower parents to co-invest and sustain a STEM program in their communities.
EDC’s Center for Children and Technology (CCT), a nonprofit research and development organization (cct.edc.org), conducted the formative evaluation of the BAS project for the last three years. Iridescent has assisted CCT researchers in the successful implementation of the evaluation (e.g., organizing site visits and meetings with partners, administering surveys, collecting consent forms). As discussed in more details below, Iridescent has always taken seriously the evaluation findings and recommendations, and has acted upon them to make program improvements. This research partnership has led
In case studies of two first-year elementary classroom teachers, we explored the influence of informal science education (ISE) they experienced in their teacher education program. Our theoretical lens was identity development, delimited to classroom science teaching. We used complementary data collection methods and analysis, including interviews, electronic communications, and drawing prompts. We found that our two participants referenced as important the ISE experiences in their development of classroom science identities that included resilience, excitement and engagement in science
MobiLLab was designed by faculty at the University of Teacher Education in St.Gallen (German: Pädagogische Hochschule St.Gallen (PHSG)) to spark interest in science in secondary school pupils in the German-speaking part of Switzerland. Now in its fourth year of operation, the mobiLLab team would like to evaluate the program’s effectiveness to inform further development. Specifically, we want to better understand how mobiLLab affects pupils’ science and technology interest, attitudes and knowledge development, and how positive changes can be sustained. The background investigation brings
DATE:
TEAM MEMBERS:
University of Teacher Education, St. Gallen, SwithzerlandRebecca Cors
This report summarizes the evaluation findings of the first year of the Science Beyond the Boundaries Early Learners Collaborative (ELC). The three-year project, funded through the Institute of Museum and Library Services (IMLS), connects science centers and children’s museums to enhance early learner programming. In Year One, the ELC brought together five institutions to collaborate directly through regularly scheduled conference call discussions. During these discussions they shared their program experience, ideas on early childhood programs, and their thoughts on current early learner
Learning to design and deliver research information customized for particular audiences is one major goal of the Museum of Science’s Research Communication Laboratory (RCL). Judging of short research presentations by an independent judge revealed that graduate students from MIT’s Center for Excitonics who participated in RCL demonstrated significantly better spoken and graphic communication skills compared with graduate students who did not experience RCL instruction. The judge rated RCL students as significantly better than non-RCL students with respect to three criteria: 1) presentation