Since the late 2000s, interest in the development and use of shared measures in the informal science, technology, engineering, and mathematics (STEM) education (ISE) field has increased. The intent is to build the capacity of evaluators to measure common outcomes of ISE experiences. We begin this chapter with a definition of shared measures, a description of related technical qualities of these measures, and a discussion of benefits and concerns around the use of shared measures. We then review recent conversations and developments around shared measures, including examples of observational
Twenty-first century skills are vital for preparing youth for careers in science, technology, engineering, and math (STEM) fields. STEM out-of-school time (OST) programs play an important role in helping youth develop these skills, particularly the teamwork skills necessary for the growing collaborative nature of STEM jobs. However, there is a lack of appropriate measures to evaluate this key programmatic outcome in STEM OST settings. This dissertation research addresses the lack of measures through the development of an instrument to assess team communication skills in middle and high school
In the last twenty years, citizen science has blossomed as a way to engage a broad range of individuals in doing science. Citizen science projects focus on, but are not limited to, nonscientists participating in the processes of scientific research, with the intended goal of advancing and using scientific knowledge. A rich range of projects extend this focus in myriad directions, and the boundaries of citizen science as a field are not clearly delineated. Citizen science involves a growing community of professional practitioners, participants, and stakeholders, and a thriving collection of
This paper explores the construction of hybrid spaces through the observation of middle school children engaged in short science projects in an informal science learning programme. Hybrid spaces are not just physical structures, but refer to contexts, relationships and knowledges developed by children as their social worlds and identities merge with the normative expectations of school science. Hybrid spaces have been characterised in three different ways: as a convergent space between academic and traditionally marginalised knowledges and discourses; as a navigational space, or a way of
This research in service to practice project will examine the impact of a 12-year statewide science field trip program called LabVenture. This hands-on program in discovery and inquiry brings middle school students and teachers across the State of Maine to the Gulf of Maine Research Institute (GMRI) in Portland, Maine to become fully immersed in explorations into the complexities of local marine science ecosystems. These intensive field trip experiences are led by informal educators and facilitated entirely within informal contexts at GMRI. Approximately 70% of all fifth and sixth grade students in Maine participate in the program each year and more than 120,000 students have attended since the program's inception in 2005. Unfortunately, little is known to date on how the program has influenced practice and learning ecosystems within formal, informal, and community contexts. As such, this research in service to practice project will employ an innovative research approach to understand and advance knowledge on the short and long-term impacts of the program within different contexts. If proven effective, the LabVenture program will elucidate the potential benefits of a large-scale field trip program implemented systemically across a community over time and serve as a reputable model for statewide adoption of similar programs seeking innovative strategies to connect formal and informal science learning to achieve notable positive shifts in their local, statewide, or regional STEM learning ecosystems.
Over the four-year project duration, the project will reach all 16 counties in the State of Maine. The research design includes a multi-step, multi-method approach to gain insight on the primary research questions. The initial research will focus on extant data and retrospective data sources codified over the 12-year history of the program. The research will then be expanded to garner prospective data on current participating students, teachers, and informal educators. Finally, a community study will be conducted to understand the potential broader impacts of the program. Each phase of the research will consider the following overarching research questions are: (1) How do formal and informal practitioners perceive the value and purposes of the field trip program and field trip experiences more broadly (field trip ontology)? (2) To what degree do short-term field trip experiences in informal contexts effect cognitive and affective outcomes for students? (3) How are community characteristics (e.g., population, distance from GMRI, proximity to the coast) related to ongoing engagement with the field trip program? (4) What are aspects of the ongoing field trip program that might embed it as an integral element of community culture (e.g., community awareness of a shared social experience)? (5) To what degree does a field trip experience that is shared by schools across a state lead to a traceable change that can be measured for those who participated and across the broader community? and (6) In what ways, if at all, can a field trip experience that occurs in informal contexts have an influence on the larger learning ecosystem (e.g., the Maine education system)? Each phase of the research will be led by a team of researchers with the requisite expertise in the methodologies and contexts required to carry out that particular aspect of the research (i.e., retrospective study, prospective study, community study). In addition, evaluation and practitioner panels of experts will provide expertise and guidance on the research, evaluation, and project implementation. The project will culminate with a practitioner convening, to share project findings more broadly with formal and informal practitioners, and promote transfer from research to practice. Additional dissemination strategies include conferences, network meetings, and peer-reviewed publications.
The potential insights this research could garner on intersectionality between formal and informal STEM learning are substantial. As a consequence, this project is co-funded by the Advancing Informal STEM Learning (AISL) and Discovery Research K-12 (DRK-12) Programs. The Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. Likewise, the Discovery Research-K12 Program seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. Informal STEM educational activities have proliferated widely in the US over the last 20 years. Additional research will further validate the long-term benefits of this mode of learning. Thus, elaborating the multitude of variables in informal learning and how those variables can be used for individual learning is yet to be defined for the circumstances of the learners. Thus, the primary objective of this work is to produce robust and detailed evidence to help shape both practice and policy for informal STEM learning in a broad array of common circumstances such as rural, urban, varying economic situations, and unique characteristics and cultures of citizen groups. Rather than pursuing a universal model of informal learning, the principal investigator will develop a series of comprehensive models that will support learning in informal environments for various demographic groups. The research will undertake a longitudinal mixed-methods approach of Out of School Time/informal STEM experiences over a five-year time span of data collection for youth ages 9-19 in urban, suburban, town, and rural communities. The evidence base will include data on youth experiences of informal STEM, factors that exert an influence on participation in informal STEM, the impact of participation on choices about educational pathways and careers, and preferences for particular types of learning activities. The quantitative data will include youth surveys, program details (e.g. duration of program, length of each program session, youth/facilitator ratio, etc.), and demographics. The qualitative data will include on-site informal interviews with youth and facilitators, and program documentation. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This guide offers insight into community engagement practices and activity development from our making and equity project, Making Connections. It includes documentation and recommendations for work that is designed to engage community partners as equal partners, and is written most of all for other practitioners.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The three-year project, Montana Models: Connecting Local and Disciplinary Practices through University-Community Partnerships, focuses on creating, implementing, and studying several learning outcomes associated with youth engagement in mathematical modeling contexts. The project builds on existing partnerships between the state's two research universities and Montana 4-H to target outreach to rural youth and bring them into a network of people who can inspire, support, and sustain STEM learning. Middle school and high school students from rural communities will be invited to a university campus for a residential modeling-based summer program l focused on mathematics and mathematical modeling. Activities at the summer program are designed to engage them in problems relevant to their own backgrounds and experiences and to honor their local funds of knowledge. The primary goal of Montana Models is to use mathematical modeling as a mechanism for bringing everyday mathematical practices already present in rural communities into contact with disciplinary practices. The project focuses on the following research questions: (1) What are the everyday mathematical practices in Montana communities? (2) How can everyday mathematical practices be leveraged and brought into contact with disciplinary practices in service of mathematizing meaningful questions within the community? (3) How do youth identify and get identified with respect to mathematics and with respect to their role in the world? (4) How does participation in project activities affect participants' knowledge of mathematical practices and content? The project uses social design experimentation, a hybrid research methodology which combines the traditions of design-based research with forms of inquiry that involve collaboration among participants, researchers, and other stakeholders, such as critical ethnography. Data sources include field notes from ethnographic observations, interviews, videos of students engaging in modeling activities, artifacts that show their mathematical work, and results from the Attitudes Towards Mathematics Inventory. Through its collaboration with 4-H, Montana Models targets outreach to rural youth across the state, especially those from groups that are typically underrepresented in STEM fields. The project is poised to impact ways in which formal and informal educators understand the knowledge bases that are already present in rural communities and how those bases may inform, support, and sustain STEM learning. Findings and deliverables will be disseminated through a public-facing website and through the 4-H infrastructure. This infrastructure includes Montana 4-H's Clover Communication Contest that will allow participating youth to showcase their projects. Research findings will be shared through local and national conferences and peer-reviewed publications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Innovations in Development project will develop and test a model for audience assessment that STEM professionals can use during their public engagement efforts. These on-the-spot assessments will allow scientists to monitor audience understanding and use this to make immediate improvements to their activity. This project fills a critical gap in the field of public science communication as expressed by outreach scientists and by the professional and academic organizations that train them. Project partners include The Astronomical Society of the Pacific, Oregon State University, Pacific Science Center, and the National Radio Astronomy Observatory. Research questions include: (1) How and under what conditions can scientists, regardless of discipline, learn to build assessment into outreach activities? (2) To what degree are scientists willing and able to change their outreach activities to include assessment to ascertain audience attentiveness or understanding? (3) To what degree will scientists be able to adjust their outreach and engagement efforts based on audience feedback, and what support do they need to do so? (4) With constraints on their time and resources, how can the model help scientists conduct audience assessment on their own? (5) Are audience members finding events more enjoyable and fruitful that use assessment techniques? Several iterative rounds of R&D will be used in the spiral design-based approach. A group of Design Testers will start each of the iterative cycles for the initial development and testing techniques, associated training, and support. Later phases will use Field Testers using a rubric for usability, validity, and reliability of various strategies. The summative evaluation of the project will begin early and add data over the life of the project. A mixed methods approach will be used including surveys of scientists once each year after they have participated in the Prototype training. Descriptive and inferential statistics will be used to document the extent to which the project has achieved its intended outcomes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This workshop is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project will conduct a two-day workshop that will gather citizen science project leaders to address barriers in citizen science research and infrastructure: The inability to holistically study the movement, engagement, persistence and learning outcomes among volunteers engaged in multiple projects. The past few years have been a time of tremendous growth in awareness of and interest in citizen science projects. The project will address an increasing gap preventing projects in three now-popular categories (apps, projects hosted on government websites, and event-based projects) from adopting the digital tools created and available through SciStarter.com. The workshop will bring citizen science project leaders together to deepen an understanding of their needs regarding the adoption of digital tools, developed by Scistarter, which will result in more comprehensive data in support of research in informal science learning outcomes of volunteers engaged in citizen science across projects and platforms. The in-person and online contributions from participants will guide the development of resources and tutorials to scale adoption.
SciStarter is a repository of hundreds of citizen science projects. Through previous NSF support, SciStarter developed digital affiliate tools which project leaders use on their own websites to enable analytics (statistics gathered from user activity online) to help projects more easily recruit and coordinator volunteers, help volunteers track their contributions across projects and platforms, and help researchers holistically study the movement and learning outcomes across projects and platforms. The proposed workshop will facilitate iteration and adoption of the tools among three classes of projects, not originally accounted for, which have dramatically increased in numbers during the past year: 1) app-based projects, 2) projects hosted on government websites, and 3) event-based projects.. By co-designing and implementing iterative versions of the tools among these projects, the project will address important gaps in research, enable a richer, more comprehensive understanding of volunteer engagement patterns, and discover opportunities to build a stronger community of citizen science practitioners who collaborate to enhance volunteer learning communities. The project will culminate in improved research in this field and improved management of citizen science projects for appropriate recruitment and retention that fosters STEM learning.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. This AISL project investigates how informal programs can broaden participation by building social capital in STEM for youth from underrepresented groups. The project integrates social network analysis with research on informal learning, and draws on a framework to connect learning across a variety of sectors. It builds on evidence that sponsorship of youth interest, affinity-based mentorship, and brokering connections to other settings and opportunities can build social capital and support interest and persistence in STEM. It represents a strategic and timely investment into research that solidifies these emerging insights from research and practice, conducting focused investigation into relational supports for STEM interests that are particularly well suited to informal programs.
The project is guided by two research questions: (1) What forms of social capital are tied to persistence in and connecting across informal STEM programs for youth from underrepresented groups? (2) What program features--specifically sponsorship, mentorship, and brokering--grow these social supports for persistence in and connecting across informal STEM programs for underserved youth? These questions are addressed through a mixed methods 18-month cross-sectional study of 200 students in three informal programs in Orange County, California that offer project-based engineering and coding programs, support mentorship, and focus on groups underrepresented in STEM. The sample will include three age categories, capturing the transition to high school, persistence during high school, and transition to college and career. Teens will be interviewed three times at 6-month intervals, spanning these transitions. The goal of this research and effort is to determine if social capital plays an extra ordinary role in learning by this group.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This project develops and examines place-based learning using mobile augmented reality experiences for rural families where museums and science centers are scarce yet where natural resources are rich with outdoor trails, parks, and forestlands. The collaborative research team, with members from rural libraries, outdoor learning centers, learning scientists at Penn State University, and rural communities in Pennsylvania, will develop augmented reality and mobile learning resources for families and children aged from 4 to 12. The goal is to help people see what is not visible in real-time in order to learn about life and earth sciences based on local watersheds, trees, and seasonal cycles that are familiar and relevant to rural communities. To accomplish this goal, the project team will create scientifically meaningful experiences for rural families and children in their out-of-school time through three iterations of research and design. Although there is evidence that augmented reality can support learning, little empirical research has been conducted to determine what makes one type of augmented learning experience more effective than others in outdoor learning spaces. This project will produce research findings on the utility of augmented reality for science learning with families and youths outdoors. This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants
Through a four-year design-based research study, researchers will investigate three research questions. (1) How can outdoor learning experiences be enhanced with augmented reality and digital resources in ways that make science more visible and interesting?; (2) How do different forms of augmentations on trails and in gardens support science learning? 3) What social roles do children and parents play in supporting each other's science learning and connections to rural communities? Data collection includes video-recordings of children and families in the outdoors, learning analytics of people's behavior, and interviews with rural families. The project's research design will allow for the development of theory, which supports rural families learning science within and about their communities. At the end of the project, the team will offer generalizable design principles for technologically-enhanced informal learning for outdoor displays, gardens, and trails.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.