Sciencenter will develop a touring exhibition, Engineer.Design.Build, to spark interest and build confidence in STEM by providing learning opportunities about the broad impact engineers have on the environment and society. The museum will partner with Cornell University's College of Engineering to develop scientific content which will be reviewed by an advisory board of representatives from the academic, business, and informal science education sectors. Partners from informal learning institutions will provide expertise on the educational content to ensure that it is accessible and engaging for the target audience of 5-11 year olds. Through a combination of focus groups, youth/guest feedback during exhibition development, and experts in girls' engagement in STEM on the advisory board, the museum will ensure that the exhibition and programming are designed to appeal to girls, and accessible to all learners. The project will include front-end, formative, and summative evaluation through observations and mediated interviews, collecting data from youth, families, and school groups.
The Massachusetts Audubon Society will develop, pilot, and implement an evaluation framework for nature-based STEM programming that serves K-12 students visiting its network of nature centers and museums. Working with an external consultant, the society will develop the framework comprised of a logic model and theory of change for fieldtrips, and develop a toolkit of evaluation data collection methodology suitable to various child development stages. The project team will design and conduct three professional development training seminars to help Massachusetts Audubon school educators develop a working understanding of the new evaluation framework for school programs and gain the skills necessary to support protocol implementation. This project will result in the development and adoption of a universal protocol to guide the collection, management, and reporting of education program evaluation data across the 19 nature centers and museums in the Massachusetts Audubon system.
This is a survey we developed in 2018 for our exploratory research study of listeners and their parents/guardians of the children's science podcast, Brains On!. The survey includes questions about who listens, when and where children listen, children's listening behaviors, motivations for listening, activities after listening, household information, and demographic questions.
This project is a Smart and Connected Communities award. The community is part of Evanston, Illinois and is composed of the lead partners described below:
EvanSTEM which is a in-school/out of school time (OST) program to improve access and engagement for students in Evanston who have underperformed or been underrepresented in STEM.
McGaw YMCA which consists of 12,000 families serving 20,000 individuals and supporting technology and makerspace activities (MetaMedia) in a safe community atmosphere.
Office of Community Education Partnerships (OCEP) at Northwestern University which provides support for the university and community to collaborate on research, teaching, and service initiatives.
This partnership will develop a new approach to learning enagement through the STEAM (Science, Technology, Engineering, Arts, and Mathematics) interests of all young people in Evanston. This project is entitled Interests for All (I4All) and builds upon existing research results of the two Principal Investigators (PIs) and previous partnerships between the lead partners (EvanSTEM and MetaMedia had OCEP as a founding partner). I4All also brings together Evanston school districts, OST prividers, the city, and Evanston's Northwestern University as participants.
In particular the project builds on PI Pinkard's Cities of Learning project and co-PI Stevens' FUSE Studios project. Both of these projects have explicit goals to broaden participation in STEAM pursuits, a goal that is significantly advanced through I4All. In this project, I4All infrastructure will be evaluated using quantitative metrics that will tell the researchers whether and to what degree Evanston youth are finding and developing their STEAM interests and whether the I4All infrastructure supports a significantly more equitable distribution of opportunities to youth. The researchers will also conduct in depth qualitative case studies of youth interest development. These longitudinal studies will complement the quantitative metrics of participation and give measures that will be used in informing changes in I4All as part of the PIs Design Based Implementation Research approach. The artifacts produced in I4All include FUSE studio projects, software infrastructure to guide the students through OST and in-school activities and to provide to the students actionable information as to logistics for participation in I4All activities, and data that will be available to all stakeholders to evaluate the effectiveness of I4All. Additionally, this research has the potential to provide for scaling this model to different communities, leveraging the OST network in one community to begin to offer professional development more widely throughout the school districts and as an exemplar for other districts. These research results could also affect strategies and policies created by local school officials and community organizations regarding how to work together to create local learning environments to create an ecosystem where formal and informal learning spaces support and reinforce STEAM knowledge.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Under the Arctic: Digging into Permafrost, a 2,000 square foot museum exhibition, engaged visitors in real and simulated experiences related to the nature of permafrost, permafrost research, and the impact of climate change on permafrost. Development of the exhibition was part of a larger National Science Foundation Advancing Informal STEM Learning grant, Hot Times in Cold Places: The Hidden World of Permafrost, awarded to the University of Alaska Fairbanks in partnership with the Oregon Museum of Science and Industry.
Two related evaluation studies led us to our conclusions. First, we
The Brains On! exploratory research study was guided by three overarching research questions:
Who is the audience for Brains On! and what are their motivations for listening to children’s science podcasts?
How are Brains On! listeners using the podcast and engaging with its content?
What kinds of impacts does Brains On! have on its audiences?
These questions were answered through a three-phase mixed-methods research design. Each phase informed the next, providing additional insights into answering the research questions. Phase 1 was a review of a sample of secondary data in the
The Farrell Fellows Summer Internship program consists of teen educators leading science, technology, engineering and math (STEM) activities for children at libraries and park locations across Chicago. The goal of this study was to learn more about the families who attend the sessions and to also look for evidence of learning and how that may be related to the moods and attitudes of the teen educators. Data was collected through observations of the sessions, pre- and post-session surveys of 26 teen educators, and 90 surveys of the parents of participating children. Field notes were coded using
The independent evaluation firm, Knight Williams, Inc., developed a two-part post-program survey to gather information about the Year 1 SciGirls CONNECT2 outreach programs conducted by 14 partner organizations. The evaluation aimed for one educator from each organization to complete Part 1 of the survey, which consisted of program reporting questions. In all, one educator from 13 partner organizations completed Part 1, for a response rate of 93%. Part 2 of the survey asked for program reflections, with a focus on perceived program goals, impacts, highlights, and challenges. Given the
The independent evaluation firm Knight Williams, Inc. administered an online survey to educators from 16 SciGirls CONNECT2 partner organizations to gather information about their anticipated use of, reflections on, and recommendations relating to the draft updated SciGirls Strategies. The evaluation aimed for two educators from each partner organization – specifically the program leader and one educator who was familiar with the original SciGirls Seven strategies – to complete the survey about the draft updated strategies after they were shared by TPT in March 2018 via an online webinar and a
The independent evaluation firm, Knight Williams, Inc., administered an online survey and conducted follow-up interviews with educators from 14 SciGirls CONNECT2 partner organizations to gather information about their use of, reflections on, and recommendations relating to the SciGirls Seven strategies. The evaluation aimed for two educators from each partner organization – specifically the program leader and one educator who was familiar with the SciGirls Seven – to share reflections on the strategies after they completed their Year 1 programs. In all, 24 educators from 13 partners completed
The National Building Museum contracted RK&A to conduct an evaluation Investigating Where We Live (IWWL), a long-running program that has brought together creative youth in the Washington, D.C. area every summer since 1996 to explore, document, and interpret the local built environment. The study goal was to examine program strengths and challenges to help NBM strategically plan for the program’s future.
How did we approach this study?
To hear a variety of perspectives on the program, RK&A conducted in-depth telephone interviews with a number of stakeholders with different
Research that seeks to understand classroom interactions often relies on video recordings of classrooms so that researchers can document and analyze what teachers and students are doing in the learning environment. When studies are large scale, this analysis is challenging in part because it is time-consuming to review and code large quantities of video. For example, hundreds of hours of videotaped interaction between students working in an after-school program for advancing computational thinking and engineering learning for Latino/a students. This project is exploring the use of computer-assisted methods for video analysis to support manual coding by researchers. The project is adapting procedures used for computer-aided diagnosis systems for medical systems. The computer-assisted process creates summaries that can then be used by researchers to identify critical events and to describe patterns of activities in the classroom such as students talking to each other or writing during a small group project. Creating the summaries requires analyzing video for facial recognition, motion, color and object identification. The project will investigate what parts of student participation and teaching can be analyzed using computer-assisted video analysis. This project is supported by NSF's EHR Core Research (ECR) program, the STEM+C program and the AISL program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. The project is funded by the STEM+Computing program, which seeks to address emerging challenges in computational STEM areas through the applied integration of computational thinking and computing activities within disciplinary STEM teaching and learning in early childhood education through high school (preK-12). As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
The video analysis systems will provide video summarizations for specific activities which will allow researchers to use these results to quantify student participation and document teaching practices that support student learning. This will support the analysis of large volumes of video data that are often time-consuming to analyze. The video analysis system will identify objects in the scene and then use measures of distances between objects and other tracking methods to code different activities (e.g., typing, talking, interaction between the student and a facilitator). The two groups of research questions are as follows. (1) How can human review of digital videos benefit from computer-assisted video analysis methods? Which aspects of video summarization (e.g., detected activities) can help reduce the time it takes to review the videos? Beyond audio analytics, what types of future research in video summarization can help reduce the time that it takes to review videos? (2) How can we quantify student participation using computer-assisted video analysis methods? What aspects of student participation can be accurately measures by computer-assisted video analysis methods? The video to be used for this study is drawn from a project focused on engineering and computational thinking learning for Latino/a students in an after-school setting. Hundreds of hours of video are available to be reviewed and analyzed to design and refine the system. The resulting coding will also help document patterns of engagement in the learning environment.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Marios PattichisSylvia Celedon-PattichisCarlos LopezLeiva