This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
The Lineage project was a collaboration between Twin Cities Public Television and the Smithsonian’s National Museum of Natural History. The project included creation of a feature-length video program, a Virtual Reality game, and a set of hands-on activities designed for use by multigenerational audiences—all of which were incorporated as part of a series of seven Fossil Festival events at museums and other sites around the United States. This report presents findings from a set of external evaluation studies that examined impacts on families who participated in Fossil Festival events as well
But many young people face signifcant economic, cultural, historical, and/or social obstacles that distance them from STEM as a meaningful or viable option— these range from under-resourced schools, race- and gender-based discrimination, to the dominant cultural norms of STEM professions or the historical uses of STEM to oppress or disadvantage socio-economically marginalized communities (Philip and Azevedo 2017). As a result, participation in STEM-organized hobby groups, academic programs, and professions remains low among many racial, ethnic, and gender groups (Dawson 2017). One solution to
The Year in ISE is a slidedoc designed to track and characterize field growth, change and impact, important publications, and current topics in ISE in 2018. Use it to inform new strategies, find potential collaborators for your projects, and support proposal development. Scope This slidedoc highlights a selection of developments and resources in 2018 that were notable and potentially useful for the informal STEM education field. It is not intended to be comprehensive or exhaustive, nor to provide endorsement. To manage the scope and length, we have focused on meta analyses, consensus reports
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Innovations in Development project will develop a collaborative program to connect and prepare researchers from divergent disciplines to communicate science to publics of all ages in three distinct informal environments: pop-ups at community events, experiential learning with problem solving events for youth, and presentations at a science museum. It will study the effects of this program on the identity and motivations of STEM researchers and evaluate the short and medium-term outcomes for public audiences. Project partners include, The STEAM Factory, OHI/O Informal Learning in Tech Program, and the Center of Science and Industry (COSI) all located in Columbus, Ohio.
The primary goals of this project are as follows. 1) Develop a collaborative program that connects and prepares researchers from divergent disciplines to creatively and effectively communicate science to people of all ages. 2) Evaluate the short and medium-term learning outcomes across different informal learning settings. 3) Study the effect of this collaborative program on the motivations and identify of the researchers. The specific research question is: In what ways do researchers motivations and identities further develop because of these collaborative experiences with divergent perspectives? A mixed method approach will be used. Data will be collected and analyzed from 3 cohorts of researchers who implement programs in different informal learning settings. A series of pre- and post interviews with scientists will be designed to answer the research question above. Interviews will be grounded in self-determination theory. Later reflective surveys will serve as triangulation for the data.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Sathya GopalakrishnanJustin MeyerRachel KajfezArnab Nandi
This CAISE report is designed to track and characterize sector growth, change and impact, important publications, hot topics/trends, new players, funding, and other related areas in Informal STEM Education (ISE) in 2017. The goal is to provide information and links for use by ISE professionals, science communicators, and interested stakeholders who want to discover new strategies and potential collaborators for project and proposal development. Designed as a slide presentation and divided into sectors, it can be used modularly or as a complete report. Each sector reports on research, events
Lineage is a comprehensive educational media and outreach initiative that will engage individuals and families in learning about deep time and evolution, helping audiences come to newfound understandings of the connections between the past, present, and future of life on Earth. The project is a partnership between Twin Cities PBS (TPT) and the Smithsonian Institution's National Museum of Natural History and is linked to the opening of that museum's new Deep Time Fossil Hall in June 2019. The project includes a two-hour film for national broadcast on PBS, and a 20-minute short version for exhibition in science centers. The documentaries will show how scientists, using paleontology, genetics, earth science and other disciplines, can reconstruct in detail the origins of living animals like birds and elephants, revealing their ancient past as well as evidence of ecological change that can inform our understanding of Earth today. Extensive educational outreach will include the creation of "Bone Hunter," an innovative VR (Virtual Reality) game designed for family co-play that engages multiple players in the process of paleontology as they piece together a fossil in a digital lab. Bone Hunter and other collaborative educational activities will be deployed at Family Fossil Festivals that will attract multi-generational learners. One such Festival will take place at the Smithsonian Institution in Washington, D.C., while others will be based at geographically diverse institutions that serve underserved rural as well as urban communities. Lineage is a collaboration between national media producers, noted learning institutions and researchers, including Twin Cities Public Television, the Smithsonian Institution / National Museum of Natural History, Schell Games, the Institute for Learning Innovation (ILI), and Rockman et al. One of the project's primary innovations is its exploration of new learning designs for families that use cutting-edge technologies (e.g. the Bone Hunter virtual reality game) and collaborative multi-generational learning experiences that advance science knowledge and inquiry-based learning. An external research study conducted by ILI will investigate how intergenerational co-play with physical artifacts compared to virtual artifacts influences STEM (Science Technology Engineering Mathematics) learning and engagement. The findings will lead to critical strategic impacts for the field, building knowledge about ongoing innovation in the free choice learning space. The project's external evaluation will be conducted by Rockman et al and evaluative findings, as well as the educational materials derived from the project, will be widely disseminated through partnerships with professional and educator groups. Clips from the Lineage film and related learning resources will be hosted on PBS LearningMedia, so educators can incorporate these resources into their classrooms, and students and lifelong learners can explore and discover on their own. The project outcomes will have broad impact on public audiences, deepening and advancing knowledge and understanding about important scientific concepts, and promoting continued, family-based collaborative learning experiences to expand and deepen STEM knowledge. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning.
Puppet interviews can be helpful for getting feedback from young children in informal learning environments like libraries, museums, or afterschool programs. While puppets are a standby for interviewing children in clinical settings and are being used more frequently in some areas of qualitative research, they tend to be under-utilized in informal learning environments - natural settings for puppets because of their connections with play (Epstein et al., 2008). Our team developed a puppet interview protocol for the Gradient research project (Gender Research on Adult-child Discussion in
This is an efficacy study through which the Denver Museum of Nature and Science, the Denver Zoo, the Denver Botanic Gardens, and three of Denver's urban school districts join efforts to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The Metropolitan Denver Urban Advantage (UA Denver) program is used for this purpose. This program consists of three design elements: (a) student-driven investigations, (b) STEM-related content, and (c) alignment of schools and informal science education institutions; and six major components: (a) professional development for teachers, (b) classroom materials and resources, (c) access to science-rich organizations, (d) outreach to families, (e) capacity building and sustainability, and (e) program assessment and student learning. Three research questions guide the study: (1) How does the participation in the program affect students' science knowledge, skills, and attitudes toward science relative to comparison groups of students? (2) How does the participation in the program affect teachers' science knowledge, skills, and abilities relative to comparison groups of teachers? and (3) How do families' participation in the program affect their engagement in and support for their children's science learning and aspirations relative to comparison families?
The study's guiding hypothesis is that the UA Denver program should improve science literacy in urban middle school students measured by (a) students' increased understanding of science, as reflected in their science investigations or "exit projects"; (b) teachers' increased understanding of science and their ability to support students in their exit projects, as documented by classroom observations, observations of professional development activities, and surveys; and (c) school groups' and families' increased visits to participating science-based institutions, through surveys. The study employs an experimental research design. Schools are randomly assigned to either intervention or comparison groups and classrooms will be the units of analysis. Power analysis recommended a sample of 18 intervention and 18 comparison middle schools, with approximately 72 seventh grade science teachers, over 5,000 students, and 12,000 individual parents in order to detect differences among intervention and comparison groups. To answer the three research questions, data gathering strategies include: (a) students' standardized test scores from the Colorado Student Assessment Program, (b) students' pre-post science learning assessment using the Northwest Evaluation Association's Measures for Academic Progress (science), (c) students' pre-post science aspirations and goals using the Modified Attitude Toward Science Inventory, (d) teachers' fidelity of implementation using the Teaching Science as Inquiry instrument, and (e) classroom interactions using the Science Teacher Inquiry Rubric, and the Reformed Teaching Observation protocol. To interpret the main three levels of data (students, nested in teachers, nested within schools), hierarchical linear modeling (HLM), including HLM6 application, are utilized. An advisory board, including experts in research methodologies, science, informal science education, assessment, and measurement oversees the progress of the study and provides guidance to the research team. An external evaluator assesses both formative and summative aspects of the evaluation component of the scope of work.
The key outcome of the study is a research-informed and field-tested intervention implemented under specific conditions for enhancing middle school science learning and teaching, and supported by partnerships between formal and informal organizations.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. The project's goal is to demonstrate an educational model fully commensurate with the demands of the 21st Century workforce, and more specifically, with the emerging “green-tech” economy.