Winterberry - Arctic Harvest Final Evaluation Report

The Arctic Harvest-Public Participation in Scientific Research (which encompasses the Winterberry Citizen Science program), a four-year citizen science project looking at the effect of climate change on berry availability to consumers has made measurable progress advancing our understanding of key performance indicators of highly effective citizen science programs. These indicators include preparation to participate in the program, alignment of goals and activities between the project scientists and participants, evidence of volunteers making a valuable contribution to science, an effective feedback loop between project staff and participants, a feeling a social and cultural importance of the project among participants, and evidence of participants extending the project in some way into a broader community.

All three of the citizen science program models being tested in the Arctic Harvest-Public Participation in Scientific Research project (a basic model, highly supported model, and highly supported storytelling model) effectively implemented four programmatic characteristics that are evidence-based good practices for engaging diverse audiences in citizen science at high levels.

• Participants in all three models were prepared to participate in a Winterberry investigation as a facilitator or contributor with both training and materials. Interviews suggest that training provided by the project prior to start-up was essential for adult participants to feel prepared and that in person on-boarding and presentations in classrooms were helpful in preparing both adult and youth participants.
• Winterberry project goals and activities aligned with, responded to, and were relevant to the needs and interests of the adult participants. Interviews indicate that most of the adult participants had curricular or learning goals related to climate change.
• Adult participants felt strongly that they were making a valuable contribution to science. Interview data indicates that most of the adult participants had a strong understanding of the ways in which the data they were collecting would be used by scientists.
• All three models established a feedback loop between the participants and the program’s science team or program staff. Interview data indicated that interactions with scientists ranged from emails and phone calls to working one-on-one with scientists to establish research sites and reviewing data with the participating youth.

Two of the six programmatic characteristics for engaging diverse audience in citizen science, social and cultural importance and community engagement, were evidenced to a greater degree among rural youth groups and groups whose educators were cross trained in a co-produced educator training program designed to facilitate climate change learning through both Indigenous and Western Science also offered by the project PIs (the Arctic and Earth SIGNs program). However, there were significant differences between participant types and program models in the implementation of these programmatic characteristics.

• Social and cultural importance rubric scores were significantly (p < .05) greater for adult participants who participated in a co-produced educator training program designed to facilitate climate change learning through both Indigenous and Western Science also offered by the project PIs (the Arctic and Earth SIGNs program) in addition to Winterberry than those adult participants who only participated in a Winterberry project.  Rural participants also reported significantly (p < .0001) higher levels of social and cultural importance of the project than did urban participants. Adult participants who identified social and cultural characteristics motivating their participation in the project explicitly provided a clear description of why their project is important to their local community in terms of food security, tradition, or climate change.
• Participants receiving the storytelling model treatment (p = .011) and who were also participating in the A&E SIGNs project (p = 0.001) were significantly more likely to report engaging their local community in project activities. Those interviewed describe engaging their local communities by bringing in Elders or local experts to help explore a community need or problem and presenting their projects and findings to a larger audience.

Data indicates that youth were engaged in the Winterberry citizen science project activities in various ways. These types of engagement included youth participating in scientific data collection, but went far beyond in the form of discourse in the field while monitoring berries, joining conference calls with scientists while on the road for youth sports because they didn’t want to miss the opportunity, data analysis, youth leadership of the project, extending the project to cooking and food preservation, and presenting the project to professional audiences. As youth engagement in citizen science projects has the potential to increase their interest in STEM, their self-efficacy for science and stewardship, and their cognitive skills to generate and apply data and engage in systems and scenarios thinking, this is a key finding. Further we found that youth engagement was significantly different for two factors: model type and participation in A&E SIGNs.

• Youth engagement was significantly higher for the storytelling (p < .001) and supported (p < .014) models than for the basic model.
• Youth engagement was significantly greater in groups whose leader had participated in A&E SIGNs in addition to Winterberry than in groups whose leader only participated in a Winterberry project (p = .007).

In addition, preliminary data suggest youth engagement is highly correlated with adult interaction, including higher levels of engagement through the supported and storytelling models, as well as engagement with scientists and Winterberry staff as evidenced by the positive feedback loop developed by the Winterberry citizen science project and by the community engagement reported by many of the adult participants interviewed.

• There was a significant positive association (p = .002) between youth engagement and the feedback loop indicator scores.
• There was a significant positive association (p = .023) between youth engagement and local community engagement indicator scores.
• There was a significant positive association (p = .036) between youth engagement and social and cultural importance indicator scores.

Youth data indicate that

• Adult preparation contributed specifically to
  • positive changes in youth STEM self-efficacy (p = 0.007) and
  • youth critical thinking scores (p = 0.039).
• Adult goal alignment contributed
  • to positive changes in youth stewardship self-efficacy (p = 0.016) and
  • negatively to youth goal alignment (p = 0.009).
• Adult identification of social and cultural important contributed
  • to positive changes in youth STEM self-efficacy (p = 0.007) and
  • to positive youth goal alignment (p = 0.007).
• Adult identification with contribution of science did not specifically contribute to any youth outcomes.
• Adult identification of a feedback loop did not specifically contribute to any youth outcomes.
• Adult identification of community engagement contributed to several youth outcomes:
  • Youth goal alignment (p < 0.001).
  • Youth report of meaningful impact (p < 0.001).
  • Youth report of communication with others about their project (p < 0.001).

Recommendations

To improve the participation in and effectiveness of citizen science across diverse audiences, particularly at high-latitudes where a high proportion of communities have populations underrepresented in STEM, we recommend the following:

• Provide in-person as well as written training materials for all participants, including youth. Update training regularly to address emerging issues and to encourage ongoing participation. This recommendation is particularly important for programs with education outcomes goals.
• Continue to highlight the important contributions of the citizen science to the research being conducted. This recommendation is particularly important for motivating adult participants.
• Encourage science and program staff to work directly with youth participants to encourage engagement, as well as to model science careers. This recommendation is particularly important for youth engagement goals.
• Share program goals for research, community impact, and youth outcomes with participants, including youth participants to help participants align their personal and professional goals with the citizen science program.
• Encourage community engagement by providing specific training to participants, including youth participants, about ways that they can engage their community. This recommendation is particularly important for engaging rural Alaska communities and underrepresented youth.
• Design programs to explicitly focus on a social and/or cultural issue for underrepresented communities. This recommendation is especially true for engaging rural Alaska communities and underrepresented youth.

If you would like to edit a resource, please email us to submit your request.