The Massachusetts Audubon Society will develop, pilot, and implement an evaluation framework for nature-based STEM programming that serves K-12 students visiting its network of nature centers and museums. Working with an external consultant, the society will develop the framework comprised of a logic model and theory of change for fieldtrips, and develop a toolkit of evaluation data collection methodology suitable to various child development stages. The project team will design and conduct three professional development training seminars to help Massachusetts Audubon school educators develop a working understanding of the new evaluation framework for school programs and gain the skills necessary to support protocol implementation. This project will result in the development and adoption of a universal protocol to guide the collection, management, and reporting of education program evaluation data across the 19 nature centers and museums in the Massachusetts Audubon system.
DATE:
-
TEAM MEMBERS:
Kris Scopinich
resourceevaluationProfessional Development, Conferences, and Networks
This document describes the summative project evaluation of 5 annual cohorts of STE(A)M teachers, mostly from California, Florida, and New Mexico participating in out-of-school authentic research experiences collecting fossils and learning about geology, biology, and the natural history along the Panama Canal, and their experiences with museums and research collections. The STEM content of this project is based on the Great American Biotic Interchange (GABI) of animals and plants across the Isthmus of Panama over the past 5 million years. This report also describes the efficacy of sustained
Brokering Youth Pathways was created to share tools and techniques around the youth development practice of “brokering” or connecting youth to future learning opportunities and resources.
This toolkit shares ways in which various out-of-school educators and professionals have approached the challenge of brokering. It provides a framework, practice briefs and reports that focus on a particular issue or challenge and provide concrete examples, as well as illustrate how project partners partners worked through designing new brokering routines in partnership with a research team.
The Discovery Research K-12 program (DR-K12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This project scales up the PBS NewsHour Student Reporting Labs (SRL), a model that trains teens to produce video reports on important STEM issues from a youth perspective. Participating schools receive a SRL journalism and digital media literacy curriculum, a mentor for students from a local PBS affiliate, professional development for educators, and support from the PBS NewsHour team. The production of news stories and student-oriented instruction in the classroom are designed to increase student learning of STEM content through student-centered inquiry and reflections on metacognition. Students will develop a deep understanding of the material to choose the best strategy to teach or tell the STEM story to others through digital media. Over the 4 years of the project, the model will be expanded from the current 70 schools to 150 in 40 states targeting schools with high populations of underrepresented youth. New components will be added to the model including STEM professional mentors and a social media and media analytics component. Project partners include local PBS stations, Project Lead the Way, and Share My Lesson educators.
The research study conducted by New Knowledge, LLC will add new knowledge about the growing field of youth science journalism and digital media. Front-end evaluation will assess students' understanding of contemporary STEM issues by deploying a web-based survey to crowd-source youth reactions, interest, questions, and thoughts about current science issues. A subset of questions will explore students' tendencies to pass newly-acquired information to members of the larger social networks. Formative evaluation will include qualitative and quantitative studies of multiple stakeholders at the Student Reporting Labs to refine the implementation of the program. Summative evaluation will track learning outcomes/changes such as: How does student reporting on STEM news increase their STEM literacy competencies? How does it affect their interest in STEM careers? Which strategies are most effective with underrepresented students? How do youth communicate with each other about science content, informing news media best practices? The research team will use data from pre/post and post-delayed surveys taken by 1700 students in the STEM Student Reporting Labs and 1700 from control groups. In addition, interviews with teachers will assess the curriculum and impressions of student engagement.
This project will advance efforts of the Innovative Technology Experiences for Students and Teachers (ITEST) program to better understand and promote practices that increase student motivations and capacities to pursue careers in fields of science, technology, engineering, or mathematics (STEM) by designing, implementing, and testing an afterschool internship program that will engage older youth in work-based learning experiences in in STEM fields. The new model program will link the resources and learning approaches of the Global Learning and Observations to Benefit the Environment (GLOBE) program to career academies where youth from populations underrepresented in STEM fields will gain direct experiences in data collection and analysis through student-led investigations in the geosciences and environmental studies. Two key outcomes of this project will be: (a) Development of a replicable model of an afterschool STEM internship program for informal STEM learning environments and schools across the nation, and (b) Development of a set of measurement tools and approaches that can assess and promote understanding regarding how youth think and feel about science and their possible future roles in science careers. Participating youth will master scientific practices and become immersed in science culture through opportunities to develop research projects, interact with scientists, and collaborate with fellow student-researchers. In the process, they will develop collaboration and communication skills, and gain an increased sense of identity and agency in science fields. They will also learn new strategies to attain their career goals.
In developing and testing the new model of an afterschool program focusing on STEM careers, the project will draw on both existing and emerging knowledge from three areas of inquiry: informal STEM learning, youth development, and work-based learning. The project will bring together theory related to work-based learning and apprenticeship to knowledge about informal STEM learning and youth development, addressing the needs of older youth as they transition to adulthood. The program will also explore the use of measurement tools that address workforce-related student learning goals in addition to social-emotional learning and STEM learning goals, adapting existing tools and developing new tools as needed. The result will be a replicable model for an afterschool, career-focused internship that facilitates STEM learning and identity, employing youth development principles, such as experiential learning, peer collaboration, adult mentoring, and meaningful contributions to the world beyond school. The project will use a mixed-methods approach to investigate four research questions: (1) What aspects of the program are most important for promoting the development of scientific practices, socio-emotional learning, and career skills? (2) How can afterschool informal science learning be designed to address the perceptions and needs of diverse groups, especially those from populations underrepresented in STEM? (3) How do youth make gains in developing facility with STEM practices, key social-emotional outcomes needed in work and civic life, and career development knowledge? And (4) How do we accurately measure development of scientific practices, socio-emotional learning and career skills? The project will develop pretest and posttest self-report measures to gauge program influence on social-emotional outcomes and career-related outcomes, and performance-based assessments and rubrics will be used to assess culminating science projects. Other factors contributing to the success of the new model will be examined through analysis of coach instructional logs, surveys, and questions, as well as participant observations, interviews, and focus groups. Project participants will be youth of ages 14-18 recruited from ten inner-city schools having large populations of students from groups underrepresented in STEM fields. Participants will meet in teams of approximately 14 interns for a total of 2.5 hours per week for 32 weeks. Each team will also meet an additional 4-6 times for weekend or overnight outings associated with their study sites.
DATE:
-
TEAM MEMBERS:
Manuel AlonsoCathy RingstaffSvetlana Darche
Twenty-first century skills are vital for preparing youth for careers in science, technology, engineering, and math (STEM) fields. STEM out-of-school time (OST) programs play an important role in helping youth develop these skills, particularly the teamwork skills necessary for the growing collaborative nature of STEM jobs. However, there is a lack of appropriate measures to evaluate this key programmatic outcome in STEM OST settings. This dissertation research addresses the lack of measures through the development of an instrument to assess team communication skills in middle and high school
The Association of Science-Technology Centers (ASTC) aims to help member institutions inform and educate the public about science through a global lens. The World Biotech Tour (WBT) was a three-year initiative designed to not only promote a greater understanding of biotechnology through public outreach and programming led by science centers and museums, but also as an opportunity for ASTC to collect useful data on the resources required and challenges encountered at different science museums around the world when implementing a long-term, multi-country program. The WBT involved students
Beyond Spaceship Earth is a multi-faceted project at The Children's Museum of Indianapolis that includes an immersive permanent exhibit about the International Space Station, the Indianan Astronaut Wall of Fame, the Schaefer Planetarium and Space Object Theater; museum theater and facilitated programs throughout the exhibit; complementary programming for schools and families in the museum's STEMLab; and a freely available unit of study for classroom teachers of 3rd-8th grades. The summative evaluation of the project used qualitative and quantitative methods including timing and tracking
The Space and Earth Informal STEM Education (SEISE) project, led by the Arizona State University with partners Science Museum of Minnesota, Museum of Science, Boston, and the University of California Berkeley’s Lawrence Hall of Science and Space Sciences Laboratory, is raising the capacity of museums and informal science educators to engage the public in Heliophysics, Earth Science, Planetary Science, and Astrophysics, and their social dimensions through the National Informal STEM Education Network (NISE Net). SEISE will also partner on a network-to-network basis with other existing coalitions and professional associations dedicated to informal and lifelong STEM learning, including the Afterschool Alliance, National Girls Collaborative Project, NASA Museum Alliance, STAR_Net, and members of the Association of Children’s Museums and Association of Science-Technology Centers. The goals for this project include engaging multiple and diverse public audiences in STEM, improving the knowledge and skills of informal educators, and encouraging local partnerships.
In collaboration with the NASA Science Mission Directorate (SMD), SEISE is leveraging NASA subject matter experts (SMEs), SMD assets and data, and existing educational products and online portals to create compelling learning experiences that will be widely use to share the story, science, and adventure of NASA’s scientific explorations of planet Earth, our solar system, and the universe beyond. Collaborative goals include enabling STEM education, improving U.S. scientific literacy, advancing national educational goals, and leveraging science activities through partnerships. Efforts will focus on providing opportunities for learners explore and build skills in the core science and engineering content, skills, and processes related to Earth and space sciences. SEISE is creating hands-on activity toolkits (250-350 toolkits per year over four years), small footprint exhibitions (50 identical copies), and professional development opportunities (including online workshops).
Evaluation for the project will include front-end and formative data to inform the development of products and help with project decision gates, as well as summative data that will allow stakeholders to understand the project’s reach and outcomes.
As the maker movement is increasingly adopted into K-12 schools, students are developing new competences in exploration and fabrication technologies. This study assesses learning with these technologies in K-12 makerspaces and FabLabs.
Our study describes the iterative process of developing an assessment instrument for this new technological literacy, the Exploration and Fabrication Technologies Instrument, and presents findings from implementations at five schools in three countries. Our index is generalizable and psychometrically sound, and permits comparison between student confidence
DATE:
TEAM MEMBERS:
Paulo BliksteinZaza KabayadondoAndrew P. MartinDeborah A. Fields
Informal science learning (ISL) organizations that are successful at providing meaningful science, technology, engineering, arts, and mathematics (STEAM) experiences for Latino children, youth, and their families share some common traits. They have leaders and staff who believe in the importance of developing culturally relevant models and frameworks that meet the needs and acknowledge the legacy of STEAM in Latino communities. Such organizations are willing to take risks to create experiences that are culturally meaningful, garner funding and implement programs by working closely with their
DATE:
TEAM MEMBERS:
Cheryl JuarezVerónika NúñezExploratorium
In 2017, Concord Evaluation Group (CEG) conducted a summative evaluation of Design Squad Global (DSG). DSG is produced and managed by WGBH Educational Foundation. WGBH partnered with FHI 360, a nonprofit human development organizations working in 70 countries, to implement DSG around the globe.
In the DSG program, children in afterschool and school clubs explored engineering through hands-on activities, such as designing and building an emergency shelter or a structure that could withstand an earthquake. Through DSG, children also had the chance to work alongside a partner club from another