The Miami Children’s Museum (MCM) contracted RK&A to conduct a summative evaluation of the Construction Zone exhibition, which was funded in part by IMLS. The evaluation focused on understanding the overall experience for walk-in visitors to the exhibition. It also explored visitor experiences with three specific exhibits with attention to problem-solving, experimentation with building materials, and collaboration.
How did we approach this study?
RK&A used two methodologies to evaluate the exhibition: interviews and focused observations. RK&A conducted 51 open-ended interviews with a
Computational Thinking (CT) is a relatively new educational focus and a clear need for learners as a 21st century skill. This proposal tackles this challenging new area for young learners, an area greatly in need of research and learning materials. The Principal Investigators will develop and implement integrated STEM+C museum exhibits and integrate CT in their existing engineering design based PictureSTEM curriculum for K-2 students. They will also pilot assessments of the CT components of the PictureSTEM curriculum. This work will make a unique contribution to the available STEM+C learning materials and assessments. There are few such materials for the kindergarten to second grade (K-2) population they will work with. They will research the effects of the curriculum and the exhibits with a mixed methods approach. First, they will collect observational data and conduct case studies to discover the important elements of an integrated STEM+C experience in both the formal in-school setting with the curriculum and in the informal out-of-school setting with families interacting with the museum exhibits. This work will provide a novel way to understand the important question of how in- and out-of-school experiences contribute to the development of STEM and CT thinking and learning. Finally, they will collect data from all participants to discover the ways that their activities lead to increases in STEM+C knowledge and interest.
The Principal Investigators will build on an integrated STEM curriculum by integrating CT and develop integrated museum exhibits. They base both activities on engineering design implemented through challenge based programming activities. They will research and/or develop assessments of both STEM+C integrated thinking and CT. Their research strategy combines Design Based Research and quantitative assessment of the effectiveness of the materials for learning CT. In the first two years of their study, they will engage in iterations on the design of the curriculum and the exhibits based on observation and case-study data. There will be 16 cases that draw from each grade level and involve data collection for the case student in both schools and museums. They will also use this work to illuminate what integrated STEM+C thinking and learning looks like across formal and informal learning environments. Based in some part on what they discover in this first phase, they will conduct the quantitative assessments with all (or at least most) students participating in the study
Informal learning institutions, such as science centers and museums, are well-positioned to broaden participation in engineering pathways by providing children from underrepresented groups with motivational, self-directed engineering design experiences. Though many informal learning institutions offer opportunities for young visitors to engage in engineering activities, little is known about the specific features of these activities that support children's motivation in engineering design processes such as problem scoping, testing, and iteration. This project will address this gap and advance foundational knowledge by identifying features of engineering design activities, as implemented within an informal setting, which support underrepresented children's engineering motivation and persistence in engineering tasks. Researchers at New York Hall of Science (NYSCI) will observe children interacting with families and museum educators as they engage in different engineering design activities in NYSCI's Design Lab, an exhibition space devoted to hands-on exploration of engineering design. They will also survey and interview the children and their caregivers about these experiences. Analyses of these data sources will result in a description of features of design activities foster motivation and task persistence in engineering design. Findings will be disseminated nationally to other informal learning institutions, which in turn can use the knowledge generated from this project to create motivational, research-based, field-tested engineering design experiences for young visitors, especially for children from underrepresented groups. The experiences may encourage children to further pursue engineering pathways, resulting in a diversified engineering workforce with the potential to drive and sustain national innovation and global technological leadership.
This project uses the framework of goal orientation, defined as learners' self-reflection of why and how they engage in tasks, to understand whether, how, and why underrepresented 7-12-year-olds engage in engineering design activities in an informal learning institution. Though previous research has suggested that goal orientation is strongly, positively related to learning and motivation in formal settings such as schools, research in informal settings has not robustly accounted for the role of goal orientation in participants' engagement with learning tasks in these unique learning environments. To better understand how children's goal orientations contribute to their motivation in engineering in informal learning institutions, researchers will answer the following research questions: (1) What are underrepresented children's goals and goal orientations while participating in engineering design activities in an informal setting? (2) What contextual factors--including facilitation strategies, materials, task relevance, and social interactions with family members--may support or discourage the adoption of different goal orientations? (3) How do goal orientations relate to children's learning experience in the engineering design activities and the likelihood that they will test and iterate their solutions? These questions will be answered through a mixed-method research study conducted with approximately 200 families, with children aged 7-12, recruited from underrepresented groups. Semi-structured clinical interviews, conducted with 20% of the children and their caregivers, as well as observations and surveys gathered from all families, will provide information on the children's goal orientation and engagement as they relate to specific engineering design activities. Qualitative content analyses and multilevel structural equation modeling will result in findings that will be disseminated widely to other institutions of informal learning. Ultimately, this project will generate new empirical knowledge regarding the features of engineering design activities in informal learning environments that increase engineering engagement and motivation among underrepresented children, thereby broadening participation in engineering pathways.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
We examined the conversational reflections of 248 families with 6–11‐year‐old children shortly after they visited a tinkering exhibit. Our aim was to understand the conditions of tinkering and conversational reflection that can enhance STEM learning opportunities for young children. We discuss implications for the design of tinkering and reflection activities that can both reveal and advance STEM learning.
The project will refine, research and disseminate making exhibits and events that the museum has developed and tested to support early engineering skill development. The project will use cardboard, a familiar and flexible material, to support the activities. The goal is to develop insights and resources for informal educators across the museum field and beyond into how to effectively structure and facilitate open-ended maker education experiences for visitors that expand the number and kinds of museums and families who can engage in these activities. Maker education is often linked to Science, Technology, Engineering and Mathematics (STEM) learning and uses hands-on and collaborative approaches to support activities and projects that foster creativity, interest, and skill development. To address patterns of inequitable access to and participation in both formal and informal learning opportunities, the project will be designed to engage families from under-represented communities and research how they participate in informal engineering activities and environments. The project will make a suite of resources available for museums and other ISE practitioners that will be developed through iterative testing at all of the different settings. These resources will be made widely available via an open access online portal.
The project will research how effectively the use of cardboard making exhibits and events engage families, particularly families from underrepresented groups, in STEM and early engineering. The project's theoretical framework combines elements of: (1) learning sciences theories of family learning in museums; (2) making as a learning process; (3) early engineering practices and dispositions, and (4) equity in museums and the maker movement. The research will be conducted within two multi-month implementations of a large-scale Cardboard Engineering gallery at the Science Museum of Minnesota and two-week scaled implementations of the gallery at each of three recruited partner museum sites. The project design interweaves evaluation and research aims. Paired observations and surveys will be used to research how effectively the project is working in different venues. This integration of research and evaluation will generate a large data set from which to generalize about cardboard making across contexts. Case studies will be used to identify barriers to engagement that can be remedied, but they will provide a rich data set for understanding family learning and engineering in making. Research findings and products will be posted on the Center for Informal Science Education website and submitted for publication in peer-reviewed journals such as Visitor Studies, ASTC Dimensions, the Journal of Pre-College Engineering Education Research and others.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This project responds to calls to increase children's exposure and engagement in STEM at an early age. With the rise of the maker-movement, the informal and formal education sectors have witnessed a dramatic expansion of maker and tinkering spaces, programs, and curricula. This has happened in part because of the potential benefits of tinkering experiences to promote access and equity in engineering education. To realize these benefits, it is necessary to continue to make and iterate design and facilitation approaches that can deepen early engagement in disciplinary practices of engineering and other STEM-relevant skills. This project will investigate how stories can be integrated into informal STEM learning experiences for young children and their families. Stories can be especially effective because they bridge the knowledge and experiences young children and their caregivers bring to tinkering as well as the conversations and hands-on activities that can extend that knowledge. In addition, a unique contribution of the project is to test the hypothesis that stories can also facilitate spatial reasoning, by encouraging children to think about the spatial properties of their emerging structures.
This project uses design-based research methods to advance knowledge and the evidence base for practices that engender story-based tinkering. Using conjecture mapping, the team will specify their initial ideas and how it will be evident that design/practices impact caregivers-child behaviors and learning outcomes. The team will consider the demographic characteristics, linguistic practices, and funds of knowledge of the participants to understand the design practices (resources, activities) being implemented and how they potentially facilitate learning. The outcome of each study/DBR cycle serves as inputs for questions and hypotheses in the next. A culturally diverse group of 300+ children ages 5 to 8 years old and their parents at Chicago Children's Museum's Tinkering Lab will participate in the study to examine the following key questions: (1) What design and facilitation approaches engage young children and their caregivers in creating their own engineering-rich tinkering stories? (2) How can museum exhibit design (e.g., models, interactive displays) and tinkering stories together engender spatial thinking, to further enrich early STEM learning opportunities? and (3) Do the tinkering stories children and their families tell support lasting STEM learning? As part of the overall iterative, design-based approach, the team will also field test the story-based tinkering approaches identified in the first cycles of DBR to be most promising.
This project will result in activities, exhibit components, and training resources that invite visitors' stories into open-ended problem-solving activities. It will advance understanding of mechanisms for encouraging engineering learning and spatial thinking through direct experience interacting with objects, and playful, scaffolded (guided) problem-solving activities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This video captures the energy and potetial of the Designing our Tomorrow project. It is intended to complement presentations and posters about Designing our Tomorrow.
The Designing Our Tomorrow project aims to develop a framework for creating exhibit-based engineering design challenges and expand an existing model of facilitation for use in engineering exhibits. Designing our Tomorrow seeks to broaden participation in engineering and build capacity within the informal science education (ISE) field while raising public awareness of the importance of sustainable engineering design practices
The Lemelson Center for the Study of Invention and Innovation (Lemelson) at the National Museum of American History (NMAH) contracted RK&A to conduct a front-end evaluation of Game Changers, a new exhibition that will highlight stories of invention and innovation related to sports. The goals of the study were to identify visitors’ associations with sports and invention and explore their reactions to preliminary exhibition concepts and titles.
How did we approach this study?
RK&A conducted 40 in-depth interviews with walk-in visitors to NMAH. As much as possible, RK&A recruited
This poster shows an overview of the The Designing Our Tomorrow (DOT) project. The project aims to develop a framework for creating exhibit-based engineering design challenges and expand an existing model of facilitation for use in engineering exhibits. DOT seeks to broaden participation in engineering and build capacity within the informal science education (ISE) field while raising public awareness of the importance of sustainable engineering design practices.
This poster was presented as part of the 2019 AISL PI Meeting. In this project, the New York Hall of Science, in collaboration with the Amazeum (Bentonville, AR), the Tech (San Jose, CA), and the Creativity Labs (Indiana University), is conducting a design-based research study to develop evidence-based guidance about how museums can use narratives to create more equitable and effective engineering experiences for girls. Through iterative activity development, the project team is exploring ways of using narrative elements (such as characters, settings, and problem frames) to communicate a story
This paper describes an NSF-funded study which explored the relationship between female-responsive exhibit designs and girls’ engagement. Across three participating science centers, 906 museum visitors ages 8 to 13 were observed at 334 interactive physics, math, engineering, and perception exhibits. We measured girls’ engagement based on whether they chose to use or return to the exhibits, opted to spend more time at them, or demonstrated deeper engagement behavior. Findings suggest that the design strategies identified in our previously developed Female-Responsive Design Framework can inform
Exploration Place, with funding from the National Aeronautics and Space Administration (NASA), contracted RK&A to conduct a summative evaluation of Design Build Fly, an exhibition and program series that explores what happens behind-the-scenes in Wichita’s aircraft plants. The goals of the study were to identify how visitors use the exhibition, explore what meaning visitors make from Design Build Fly, and understand to what extent visitors’ meaning-making aligns with intended outcomes.
How did we approach this study? RK&A conducted timing and tracking observations of visitors to the