In the last twenty years, citizen science has blossomed as a way to engage a broad range of individuals in doing science. Citizen science projects focus on, but are not limited to, nonscientists participating in the processes of scientific research, with the intended goal of advancing and using scientific knowledge. A rich range of projects extend this focus in myriad directions, and the boundaries of citizen science as a field are not clearly delineated. Citizen science involves a growing community of professional practitioners, participants, and stakeholders, and a thriving collection of
This summative evaluation report presents findings and lessons learned on the STEM Ambassadors project. The STEM Ambassadors project, funded by the National Science Foundation’s (NSF) Advancing Informal STEM Learning (AISL) program, combines elements of existing programs designed to engage STEM professionals with the public around their scientific research to create a new model for public engagement of science. This new model recruits, provides training for, and assists STEM professionals in drawing on their own interests, hobbies and backgrounds to connect with audiences that may not have
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Innovations in Development project will develop and test a model for audience assessment that STEM professionals can use during their public engagement efforts. These on-the-spot assessments will allow scientists to monitor audience understanding and use this to make immediate improvements to their activity. This project fills a critical gap in the field of public science communication as expressed by outreach scientists and by the professional and academic organizations that train them. Project partners include The Astronomical Society of the Pacific, Oregon State University, Pacific Science Center, and the National Radio Astronomy Observatory. Research questions include: (1) How and under what conditions can scientists, regardless of discipline, learn to build assessment into outreach activities? (2) To what degree are scientists willing and able to change their outreach activities to include assessment to ascertain audience attentiveness or understanding? (3) To what degree will scientists be able to adjust their outreach and engagement efforts based on audience feedback, and what support do they need to do so? (4) With constraints on their time and resources, how can the model help scientists conduct audience assessment on their own? (5) Are audience members finding events more enjoyable and fruitful that use assessment techniques? Several iterative rounds of R&D will be used in the spiral design-based approach. A group of Design Testers will start each of the iterative cycles for the initial development and testing techniques, associated training, and support. Later phases will use Field Testers using a rubric for usability, validity, and reliability of various strategies. The summative evaluation of the project will begin early and add data over the life of the project. A mixed methods approach will be used including surveys of scientists once each year after they have participated in the Prototype training. Descriptive and inferential statistics will be used to document the extent to which the project has achieved its intended outcomes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. Using community-based participatory research, human-centered design, and developmental evaluation approaches, this project will co-create a national community science project, which is a form of Public Participation in STEM Research (PPSR) or citizen science. The project seeks to increase inclusion and equity in STEM, while building capacity in research and science through leadership and participation by both historically underrepresented communities and established academic institutions. Participants will record their levels of perceived stress and well-being in relation to their local soundscape. This project will help to connect participants, specifically youth and their families from communities underrepresented and underserved in STEM, with science and nature through learning that is authentic and relevant to their lives. With support of a mobile app, community science project participants explore and map soundscapes to identify areas of "excessive noise" as well as "noise refuges." Community-led dissemination of results will inform adoption of community science projects by other community-based organizations. The project is a collaboration involving fifteen community leaders, the Cornell Lab of Ornithology, Metro Atlanta Urban Farm in Atlanta, WorldBeat Center in San Diego, CLUES in St. Paul, MN, and Camp Compass in Allentown, PA. PPSR has exciting potential to engage broad audiences in STEM learning and research, particularly when projects are genuinely co-created by local communities, scientists, and educators. Using a community-based participatory research approach in the project fosters direct involvement from community members in integration of science practices and learning, from the development of the research questions, design of the study, to analysis, interpretation, and dissemination of results. Human-centered design is important as it is an established collaborative approach for addressing real-world problems facing human communities. In this project, human-centered design will include technology in the problem-solving phases of inspiration, ideation, and implementation. Developmental evaluation will provide on-going direct feedback to the project team over the project lifespan with respect to equity and inclusion in PPSR, an informal STEM learning environment. The project will further knowledge and understanding of soundscapes and noise pollution, enable use of mobile apps to map soundscapes in urban environments, and contribute to the relationship of noise to well-being.The project will produce conceptual frameworks and practical resources (toolkits) to guide future community-based PPSR efforts. Results from this project will advance knowledge, skills, attitudes, and behaviors of PPSR practitioners regarding equity, diversity, and inclusion of underrepresented audiences in STEM. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Karen PurcellBobby WilsonMakeda CheatomJohn AnnoniTanya SchuhDavid BonterRick Bonney
resourceresearchProfessional Development, Conferences, and Networks
The Rural Informal STEM Learning Conference, sponsored by the National Science Foundation (NSF) and held at its headquarters on September 13–14, 2018, was the first of its kind to bring together key innovators and experts in rural STEM learning outside of school. People who live in rural settings are a frequently overlooked and significantly under-represented STEM audience. At the conference, which was led by the Maine Mathematics and Science Alliance, we addressed this key question: How can we build on recent innovations to create more effective experiences and pathways for informal (out-of
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This Innovations in Development project will develop a collaborative program to connect and prepare researchers from divergent disciplines to communicate science to publics of all ages in three distinct informal environments: pop-ups at community events, experiential learning with problem solving events for youth, and presentations at a science museum. It will study the effects of this program on the identity and motivations of STEM researchers and evaluate the short and medium-term outcomes for public audiences. Project partners include, The STEAM Factory, OHI/O Informal Learning in Tech Program, and the Center of Science and Industry (COSI) all located in Columbus, Ohio.
The primary goals of this project are as follows. 1) Develop a collaborative program that connects and prepares researchers from divergent disciplines to creatively and effectively communicate science to people of all ages. 2) Evaluate the short and medium-term learning outcomes across different informal learning settings. 3) Study the effect of this collaborative program on the motivations and identify of the researchers. The specific research question is: In what ways do researchers motivations and identities further develop because of these collaborative experiences with divergent perspectives? A mixed method approach will be used. Data will be collected and analyzed from 3 cohorts of researchers who implement programs in different informal learning settings. A series of pre- and post interviews with scientists will be designed to answer the research question above. Interviews will be grounded in self-determination theory. Later reflective surveys will serve as triangulation for the data.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Sathya GopalakrishnanJustin MeyerRachel KajfezArnab Nandi
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. In this Innovations and Development project, Child Trends, in collaboration with Ivanhoe Broadcast News, will expand the reach of the Child Trends News Service, and rigorously evaluate its impact on viewers. The News Service aims to build the public's knowledge of, and appreciation for, social science research and to encourage adoption of research-informed parenting practices associated with positive child development--particularly among Latino parents. First produced in 2017 through a NSF proof of concept grant, the Child Trends News Service covers actionable, child-focused, social science research. By featuring this research on local TV news, the project expands access to evidence-based parenting recommendations. As of February 2018, 89 stations had subscribed to the News Service, including eight stations in the top 25 Latino-serving TV markets that reach 38% of all Hispanic TV Households in those 25 markets. This project is a response to the challenges faced by U.S. children, of whom more than one in five live in poverty. The focus on Latino parents is in response Latinos' increasing share of all children, and that Latino children are disproportionately poor, in comparison to their peers. The project will examine the impact of the News Service on parents who view the news reports in their homes, as well as Latino parents viewing the News Service as part of their participation in the Abriendo Puertas (Opening Doors) community-based parenting program. This research will contribute to the knowledge base of what we know about how people access and use science, technology, engineering and math (STEM) information across settings.
The overarching aim of this project is to leverage commercial television news to reach populations who have historically been underrepresented in STEM education and careers. The goals of the project are to:
1. Build and manage an interdisciplinary collaborative, including news media professionals, researchers, practitioners in organizations serving at-risk families, and experts in STEM communications and Latino studies.
2. Leverage mass media news outlets to deliver social science research on children to at-risk populations, with a focus on reaching Latino parents.
3. Advance the field of informal STEM learning by exploring how the public interacts with actionable research on child development to inform their knowledge, attitudes and behaviors.
4. Expand the reach and application of the news products through strategic outreach to other stakeholders in the child development field including programs serving under-served families.
To accomplish these goals, the project will further strengthen an Advisory Panel to inform content development, study design, interpretation of findings, dissemination of study results, and the transition of the project after the NSF grant period. The project will continue to provide eight (both in English and Spanish) stories each month to TV stations and strategically grow the reach in top Latino markets. The editorial process will be informed by surveys of Latino parents to identify topics of interest. Through a random-assignment impact study with local TV news audiences from diverse racial/ethnic groups, the project will evaluate the impact of the News Service. The project will use formative research methods to refine messaging and examine the potential for repurposing the videos through a parenting program for Latino parents.
The Child Trends News Service seeks broader impacts in three areas: increasing the public's scientific literacy and engagement with science and technology; increasing partnerships between academia, industry, and others; and improving the well-being of individuals in society.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
The Implementation Manual is a practical guide for organizations planning to implement Portal to the Public. It represents the experiences of Portal to the Public sites and is supported by research and evaluation results. The seven chapters in the manual are designed around the main components of the Portal to the Public guiding framework. The chapters guide you through the process of developing successful conversation-based public programs featuring scientists.
Chapter 1: The Guiding Framework
Chapter 2: Conceptual Planning
Chapter 3: Partnership and Relationship Building
Chapter
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will build and test a new model for co-created public engagement with science activities in partnership with civic, community, and scientist partners. The innovation to be tested is deliberative dialogues in science museums that help reduce the polarization about socio-scientific issues, giving people a greater voice in science, and addressing barriers that disconnect scientists from the public. The project will engage four target audiences (informal science education/ISE professionals, civic, community and scientist partners). Science museum partners include Museum of Science (MOS) Boston, Oregon Museum of Science, the Michigan Science Center, and the North Carolina Museum of Life and Science. The project is designed to have a strategic impact on how ISE institutions choose topics of STEM engagement and build public Forum programs.
There will be two evaluation teams for the project. MOS Research and Evaluation will act as formative evaluation mentors for the four partner sites as they co-create their forums. They will provide evaluation capacity building for the sites using team-based inquiry as they create and understand the potential impacts and outcomes of the model. Data collection will include panel surveys and focus groups. The evaluation will explore how the forums can decrease 1) public polarization around STEM topics and (2) the disconnect between scientists, civic organization, and the public. The external summative evaluation will be conducted by Rockman et al (REA). They will conduct a study of the project's process to help the team identify challenges and strategies for overcoming them as they work through the phases of public engagement. The summative study will focus on the project goals taking a qualitative approach. Early interviews with partner participants will explore their strengths and weaknesses in taking on this type of public engagement model including the extent of their previous work with civic partners. Later interviews will investigate what factors have enabled or hindered this project. Summative evaluation questions will also address changes in attitudes toward public engagement with science. REA will collect feedback from summit attendees through intercept interviews and post-surveys administered within a week at the event's conclusion to explore the any changes in knowledge or confidence in undertaking this type of model. REA will present findings from the external evaluation during the annual meeting of the Association of Science-Technology Centers and publish reports to Informalscience.org. Once the model has been developed and tested it will be disseminated to an initial group of 25-30 science museums and eventually to the entire ISE museum field.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Many of the nation's poorest-performing schools are in rural areas. Anecdotal information suggests participation in and access to informal STEM learning opportunities in Mississippi - a state with among the lowest STEM-career readiness in the nation - is unequally distributed among geographic regions and sociocultural environments. Informal learning programs in science, technology, engineering, and math (STEM) have the potential to reach into rural communities and provide a bridge to greater STEM access, literacy, and career readiness. Building Bridges: Broadening the STEM Conversation in Rural Mississippi will initiate a dialog among key practitioners, experts, and stakeholders in informal STEM learning focused on identifying the causes of and solutions to STEM inclusion barriers among rural youth. The goal of this Advancing Informal STEM Learning (AISL) Conference Project aligns with NSF's mission to promote the progress of science for all segments of society, including rural K-12 students. Solutions to STEM disconnections identified in Mississippi through this project will have relevance and transferability to rural communities across the southeastern US, given regional commonalities in socioeconomic, educational, and cultural factors.
This project aims to conduct an interactive and participant-based summit that brings together key leaders and experts from informal science learning institutions and organizations, STEM-related agencies and industries, and rural community groups to improve methods for linking informal STEM learning opportunities with rural, K-12 students. The goal of the project is to identify the common barriers and explore potential solutions to informal STEM participation by rural K-12 students in Mississippi. With the guidance of a steering committee, a Mississippi STEM Consortium will be formed and convened at a 2019 Mississippi Informal STEM Consortium Summit with the following goals: (1) Identify broad barriers to informal STEM learning in diverse and rural K-12 populations. (2) Define crucial and transformative elements in informal STEM programs deemed successful in rural student recruitment and engagement. (3) Improve collaborative networking to enhance the role of informal education in building statewide STEM capacity. These objectives will be met by developing, implementing, and evaluating statewide needs-assessment surveys and a two-day summit of Consortium members. The project evaluator will ensure process and outcome evaluations are properly conducted throughout the entire course of the project to inform planning, promote iterative improvement, monitor progress, and ensure achievement of desire objectives. With regards to broader impacts, it is anticipated that outcomes from this project will have impact within and beyond Mississippi's borders. Expected project outcomes include scientific manuscripts on needs-assessment surveys, modified approaches to existing informal STEM activities, future research on identified informal STEM participation barriers and mitigation measures, new collaborations that broaden participation and expand future research, and a draft Informal STEM Strategic Plan for Mississippi. Varied dissemination methods will be used to communicate the findings broadly.
This conference project is funded by the AISL program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Leslie BurgerSarah LeeKatherine EcholsVemitra White
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understanding of deeper learning by participants. This pilot study, Akeakamai (Hawaiian, literally lover of wisdom, scientist, scholar), will explore the convergence of contemporary Western science topics with indigenous Hawaiian culture-based science experiences as a mechanism to strengthen STEM perceptions, cross-cultural science collaboration, and multi-generational community engagement with STEM. The work is grounded in the notion that STEM learning within the context of local informal indigenous community settings should be culturally responsive and culturally sustaining, and should privilege indigenous epistemologies. If successful, the results of this pilot could provide valuable insights on effective approaches to developing and implementing culturally consistent and sustainable multigenerational STEM engagement among Native Hawaiians and Pacific Islanders, and across the Pacific region.
Over a two-year duration, the study will address three research questions. (1) To what extent does inclusion of culture into curriculum designed for informal Culture-Science Explorations mitigate perceived barriers to participation in science? (2) What barriers do community members perceive to limit their participation in science? (3) What are the areas of consonance between Native Hawaiian and Western scientific approaches to knowledge and learning? Approximately 200 predominantly Native Hawaiian and Pacific Islanders, ranging in age from 8 - 85 years old, will participate in the pilot. The research team will collect participant data during all phases of the social intervention, a suite of culture-science exploration experiences held at the Halau Inana, a Native Hawaiian community collaboration space. The intervention will employ pedagogical methods that are responsive to Hawaiian cultural norms to deliver content that integrates across the interfaces of Western science and technology and indigenous knowledge, and incorporates Hawaiian language. A rigorous external evaluation will also be conducted. The results of the research and evaluation will be broadly disseminated. Ultimately, the project aims to develop a conceptual and practical cross-cultural, multi-generational framework for community-based science learning in Hawai'i that can serve as a model for future research and programs that extend into and beyond indigenous communities of the Pacific region.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Helen TurnerJonathan BakerChrystie Naeole
resourceprojectProfessional Development, Conferences, and Networks
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project is a two-day conference, along with pre- and post-conference activities, with the goal of furthering the informal science learning field's review of the research and development that has been conducted on data visualizations that have been used to help the public better understand and become more engaged in science. The project will address an urgent need in informal science education, providing a critical first step towards a synthesis of research and technology development in visualization and, thus, to inform and accelerate work in the field in this significant and rapidly changing domain.
The project will start with a Delphi study by the project evaluator prior to the conference to provide an Emerging Field Assessment on data visualization work to date. Then, a two-day conference at the Exploratorium in San Francisco and related activities will bring together AISL-funded PIs, computer scientists, cognitive scientists, designers, and technology developers to (a) synthesize work to date, (b) bring in relevant research from fields outside of informal learning, and (c) identify remaining knowledge gaps for further research and development. The project team will also develop a website with videos of all presentations, conference documentation, resources, and links to social media communities; and a post-conference publication mapping the state of the field, key findings, and promising technologies.
The initiative also has a goal to broaden participation, as the attendees will include a diverse cadre of professionals in the field who contribute to data visualization work.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.