Inverness Research has been the external evaluator for CAISE for the past eight years. The year 8 and year 9 report, along with a brief executive summary, highlights findings from the external evaluation of CAISE for this award period, along with comparison data gathered from the previous years’ evaluation efforts. Our evaluation findings show that CAISE is important infrastructure for the ISE field, addressing important needs in the field through resources that are well known, high-quality, widely used, well-designed, accessible, valued and trustworthy.
This evaluation report highlights finding on the evaluation of the Center for Advancement of Informal Science Education (CAISE), a cooperative agreement with NSF, is a partnership of the Association of Science-Technology Centers with faculty and professionals from the University of Pittsburgh Center for Learning in Out-of-School Environments, Oregon State University, Science Museum of Minnesota, Visitor Studies Association, KQED Public Media, Advisors and other collaborators. CAISE is working to support ongoing improvement of, and NSF investments in, the national infrastructure for informal
The University of Washington, the Exploratorium, the Education Development Center, Inverness Research, and the University of Colorado - Boulder have come together to form a Research+Practice (R+P) Collaboratory. The Collaboratory seeks to address and reframe the gap between research and practice in K-12 STEM education. This gap persists despite decades of work by many leading organizations, associations, and individuals. Attempts to close the gap have generally focused on creating resources and mechanisms that first explain or illustrate "what research says" and then invite educators to access and integrate findings into practice. Recently, however, attention has turned to the ways in which the medical sciences are addressing the gap between research and clinical practice through the developing field of "translational research." In medicine, the strategy has been to shift the focus from adoption to adaptation of research into practice. Implicit in the notion of adaptation is a bi-directional process of cultural exchange in which both researchers and practitioners come to understand how the knowledge products of each field can strengthen the professional activities in the other. Along these lines, the R+P Collaboratory is working with leading professional associations and STEM improvement efforts to leverage their existing knowledge and experience and to build sustainable strategies for closing the gap. Activities include:
Collecting, creating and synthesizing translational research resources to expand STEM educators' and educational leaders' access and awareness to current relevant research.
Supporting multiple opportunities for cross-sector (research and practice; education and social sciences; formal and informal) meetings to foster critical engagement and cultural exchange.
Testing, documenting and innovating new resources and mechanisms at Adaptation Sites and disseminating both products and results through the R+P Resource Center.
The R+P Collaboratory is developing an online 'Go-To' Resource Center website that houses the resources collected, created, and curated by the Collaboratory. The Resource Center also has significant 'Take-Out' features, with all materials meta-tagged so that they can be automatically uploaded, reformatted, and integrated into the existing communication and professional development mechanisms (e.g., newsletters, digests, conferences, and websites) of a dozen leading professional associations within a Professional Association Partner Network.
In light of new and emerging standards in the STEM disciplines, the Collaboratory is focusing its work on four salient and timely bodies of research: (a) STEM Practices, (b) Formative Assessment, (c) Cyberlearning, and (d) Learning as a Cross-Setting Phenomenon. Special emphasis is being placed on research and practice that focuses on the learning of children and youth from communities historically underrepresented in STEM fields.
The work of the R+P Collaboratory includes research and evaluation of its own efforts through studies aimed at answering the following questions:
How are Collaboratory resources and engagement activities accessed, experienced and leveraged by participants?
What resources, mechanisms and learning contexts support cultural exchange among STEM education researchers and practitioners?
What new kinds of practices result when research-based evidence is adapted into evidence-based practices, and how does it change learning opportunities for K-12 aged children?
How can effective strategies, mechanisms and resources of the Collaboratory be scaled and adapted to new contexts?
The Computational Thinking in Ecosystems (CT-E) project is funded by the STEM+Computing Partnership (STEM+C) program, which seeks to advance new approaches to, and evidence-based understanding of, the integration of computing in STEM teaching and learning. The project is a collaboration between the New York Hall of Science (NYSCI), Columbia University's Center for International Earth Science Information Network, and Design I/O. It will address the need for improved data, modeling and computational literacy in young people through development and testing of a portable, computer-based simulation of interactions that occur within ecosystems and between coupled natural and human systems; computational thinking skills are required to advance farther in the simulation. On a tablet computer at NYSCI, each participant will receive a set of virtual "cards" that require them to enter a computer command, routine or algorithm to control the behavior of animals within a simulated ecosystem. As participants explore the animals' simulated habitat, they will learn increasingly more complex strategies needed for the animal's survival, will use similar computational ideas and skills that ecologists use to model complex, dynamic ecological systems, and will respond to the effects of the ecosystem changes that they and other participants elicit through interaction with the simulated environment. Research on this approach to understanding interactions among species within biological systems through integration of computing has potential to advance knowledge. Researchers will study how simulations that are similar to popular collectable card game formats can improve computational thinking and better prepare STEM learners to take an interest in, and advance knowledge in, the field of environmental science as their academic and career aspirations evolve. The project will also design and develop a practical approach to programing complex models, and develop skills in communities of young people to exercise agency in learning about modeling and acting within complex systems; deepening learning in young people about how to work toward sustainable solutions, solve complex engineering problems and be better prepared to address the challenges of a complex, global society.
Computational Thinking in the Ecosystems (CT-E) will use a design-based study to prototype and test this novel, tablet-based collectable card game-like intervention to develop innovative practices in middle school science. Through this approach, some of the most significant challenges to teaching practice in the Next Generation Science Standards will be addressed, through infusing computational thinking into life science learning. CT-E will develop a tablet-based simulation representing six dynamic, interconnected ecosystems in which students control the behaviors of creatures to intervene in habitats to accomplish goals and respond to changes in the health of their habitat and the ecosystems of which they are a part. Behaviors of creatures in the simulation are controlled through the virtual collectable "cards", with each representing a computational process (such as sequences, loops, variables, conditionals and events). Gameplay involves individual players choosing a creature and habitat, formulating strategies and programming that creature with tactics in that habitat (such as finding food, digging in the ground, diverting water, or removing or planting vegetation) to navigate that habitat and survive. Habitats chosen by the participant are part of particular kinds of biomes (such as desert, rain forest, marshlands and plains) that have their own characteristic flora, fauna, and climate. Because the environments represent complex dynamic interconnected environmental models, participants are challenged to explore how these models work, and test hypotheses about how the environment will respond to their creature's interventions; but also to the creatures of other players, since multiple participants can collaborate or compete similar to commercially available collectable card games (e.g., Magic and Yu-Go-Oh!). NYSCI will conduct participatory design based research to determine impacts on structured and unstructured learning settings and whether it overcomes barriers to learning complex environmental science.
The Discovery Research K-12 program (DR-K12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This project scales up the PBS NewsHour Student Reporting Labs (SRL), a model that trains teens to produce video reports on important STEM issues from a youth perspective. Participating schools receive a SRL journalism and digital media literacy curriculum, a mentor for students from a local PBS affiliate, professional development for educators, and support from the PBS NewsHour team. The production of news stories and student-oriented instruction in the classroom are designed to increase student learning of STEM content through student-centered inquiry and reflections on metacognition. Students will develop a deep understanding of the material to choose the best strategy to teach or tell the STEM story to others through digital media. Over the 4 years of the project, the model will be expanded from the current 70 schools to 150 in 40 states targeting schools with high populations of underrepresented youth. New components will be added to the model including STEM professional mentors and a social media and media analytics component. Project partners include local PBS stations, Project Lead the Way, and Share My Lesson educators.
The research study conducted by New Knowledge, LLC will add new knowledge about the growing field of youth science journalism and digital media. Front-end evaluation will assess students' understanding of contemporary STEM issues by deploying a web-based survey to crowd-source youth reactions, interest, questions, and thoughts about current science issues. A subset of questions will explore students' tendencies to pass newly-acquired information to members of the larger social networks. Formative evaluation will include qualitative and quantitative studies of multiple stakeholders at the Student Reporting Labs to refine the implementation of the program. Summative evaluation will track learning outcomes/changes such as: How does student reporting on STEM news increase their STEM literacy competencies? How does it affect their interest in STEM careers? Which strategies are most effective with underrepresented students? How do youth communicate with each other about science content, informing news media best practices? The research team will use data from pre/post and post-delayed surveys taken by 1700 students in the STEM Student Reporting Labs and 1700 from control groups. In addition, interviews with teachers will assess the curriculum and impressions of student engagement.
“Monkeying Around: Digital Media and Parent/Child Engagement Resources to Increase Preschool Computational Thinking” is a new project that uses animation, live-action videos, and hands-on activities to support joint engagement of children and caregivers around computational thinking concepts and practices. WGBH, a leading producer of educational STEM media, developed prototypes of videos and hands-on activities around the project’s computational thinking learning goals for young children. Education Development Center (EDC), WGBH’s research partner for the project, conducted a small formative
Techbridge Girls’ mission is to help girls discover a passion for science, engineering, and technology (SET). In August 2013, Techbridge Girls was awarded a five-year National Science Foundation grant to scale up its after-school program from the San Francisco Bay Area to multiple new locations around the United States. Techbridge Girls began offering after-school programming at elementary and middle schools in Greater Seattle in 2014, and in Washington, DC in 2015.
Education Development Center is conducting the formative and summative evaluation of the project. To assess the
Summative evaluation of the Marcellus Matters: EASE project. Marcellus Matters: Engaging Adults in Science and Energy (EASE) was a program of Penn State University’s Marcellus Center for Outreach and Research (MCOR), in collaboration with other experts across the university. The first year of program activities took place in 2012, and the project continued through September 2016. EASE was a multidisciplinary initiative that provided adults in rural Pennsylvania with opportunities to increase their knowledge of science and energy systems and engage in scientific inquiry and investigation
The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.
The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.
The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”
DATE:
-
TEAM MEMBERS:
Maritza MacdonaldMeryle WeinsteinRosamond KinzlerMordecai-Mark Mac LowEdmond MathezDavid Silvernail
RMC Research designed evaluation activities to provide formative and summative feedback to the Hubbard Brook Research Foundation (HBRF) on their NSF Pathways project, Forest Science Dialogues (FSD). FSD consists of a plan to engage with scientists at the Hubbard Brook Ecosystem Study and with the surrounding community using the Hubbard Brook Roundtable dialogue process in order to facilitate mutual learning. The purpose of this engagement was to increase public knowledge, understanding, and awareness of ecosystem science in the Northern Forest in order to enrich local dialogue surrounding