This document is the final summative evaluation report written by EDC, the external evaluator of the STEM Guides project. The report concludes that the project was highly ambitious, with many dynamic and evolving pieces. It was deemed successful as a model of brokering connections between students aged 10-18 and STEM resources and opportunities in rural Maine communities. The STEM Guides program contributed to the increase in STEM awareness within each community, as well as connecting youth with interesting and relevant STEM experiences.
Identifying private gardens in the U.K. as key sites of environmental engagement, we look at how a longer-term online citizen science programme facilitated the development of new and personal attachments of nature. These were visible through new or renewed interest in wildlife-friendly gardening practices and attitudinal shifts in a large proportion of its participants. Qualitative and quantitative data, collected via interviews, focus groups, surveys and logging of user behaviours, revealed that cultivating a fascination with species identification was key to both ‘helping nature’ and wider
DATE:
TEAM MEMBERS:
Nirwan SharmaSam GreavesAdvaith SiddharthanHelen AndersonAnnie RobinsonLaura Colucci-GrayAgung Toto WibowoHelen BostockAndrew SalisburyStuart RobertsDavid SlawsonRené van der Wal
Online citizen science platforms increasingly provide types of infrastructural support previously only available to organisationally-based professional scientists. Other practices, such as creative arts, also exploit the freedom and accessibility afforded by the World Wide Web to shift the professional-amateur relationship. This paper compares communities from these two areas to show that disparate practices can learn from each other to better understand their users and their technology needs. Three major areas are discussed: mutual acknowledgement, infrastructural support, and platform
We developed a multi-touch interface for the citizen science video game Foldit, in which players manipulate 3D protein structures, and compared multi-touch and mouse interfaces in a 41-subject user study. We found that participants performed similarly in both interfaces and did not have an overall preference for either interface. However, results indicate that for tasks involving guided movement to dock protein parts, subjects using the multi-touch interface completed tasks more accurately with fewer moves, and reported higher attention and spatial presence. For tasks involving direct
DATE:
TEAM MEMBERS:
Thomas MuenderSadaab Ali GulaniLauren WestendorfClarissa VerishRainer MalakaOrit ShaerSeth Cooper
The Miami Children’s Museum (MCM) contracted RK&A to conduct a summative evaluation of the Construction Zone exhibition, which was funded in part by IMLS. The evaluation focused on understanding the overall experience for walk-in visitors to the exhibition. It also explored visitor experiences with three specific exhibits with attention to problem-solving, experimentation with building materials, and collaboration.
How did we approach this study?
RK&A used two methodologies to evaluate the exhibition: interviews and focused observations. RK&A conducted 51 open-ended interviews with a
Effective classification of large datasets is a ubiquitous challenge across multiple knowledge domains. One solution gaining in popularity is to perform distributed data analysis via online citizen science platforms, such as the Zooniverse. The resulting growth in project numbers is increasing the need to improve understanding of the volunteer experience; as the sustainability of citizen science is dependent on our ability to design for engagement and usability. Here, we examine volunteer interaction with 63 projects, representing the most comprehensive collection of online citizen science
DATE:
TEAM MEMBERS:
Helen SpiersAlexandra SwansonLucy FortsonBrooke SimmonsLaura TrouilleSamantha BlickhanChris Lintott
In citizen science, user-centred development is often emphasised for its potential to involve participants in the development of technology. We describe the development process of the mobile app “Naturblick” as an example of a user-centred design in citizen science and discuss digital user feedback with regard to the users' involvement. We have identified three types of digital user feedback using qualitative content analysis: general user feedback, contributory user feedback and co-creational user feedback. The results indicate that digital user feedback can link UCD techniques with more
Although hundreds of citizen science applications exist, there is lack of detailed analysis of volunteers' needs and requirements, common usability mistakes and the kinds of user experiences that citizen science applications generate. Due to the limited number of studies that reflect on these issues, it is not always possible to develop interactions that are beneficial and enjoyable. In this paper we perform a systematic literature review to identify relevant articles which discuss user issues in environmental digital citizen science and we develop a set of design guidelines, which we evaluate
DATE:
TEAM MEMBERS:
Artemis SkarlatidouAlexandra HamiltonMichalis VitosMuki Haklay
We discuss the potential application to virtual citizen science of a recent standard (BS ISO 27500:2016 “The human-centred organisation”) which encourages the adoption of a sociotechnical systems perspective across a wide range of businesses, organizations and ventures. Key tenets of the standard concern taking a total systems approach, capitalizing on individual differences as a strength, making usability and accessibility strategic objectives, valuing personnel and paying attention to ethical and values-led elements of the project in terms of being open and trustworthy, social responsibility
DATE:
TEAM MEMBERS:
Robert HoughtonJames SprinksJessica WardlawSteven BamfordStuart Marsh
The growing interest in citizen science has resulted in a new range of digital tools that facilitate the interaction and communications between citizens and scientists. Considering the ever increasing number of applications that currently exist, it is surprising how little we know about how volunteers interact with these technologies, what they expect from them, and why these technologies succeed or fail. Aiming to address this gap, JCOM organized this special issue on the role of User Experience (UX) of digital technologies in citizen science which is the first to focus on the qualities and
DATE:
TEAM MEMBERS:
Artemis SkarlatidouMarisa PontiJames SprinksChristian NoldMuki HaklayEiman Kanjo
RK&A conducted a summative evaluation of the New Glass Now exhibition at the Corning Museum of Glass. The interpretative goal for the exhibition is to challenge visitors to expand their notions of glass in contemporary art and design.
Methodology
The goal of the study is to explore visitors’ experiences in the temporary exhibition—the first exhibition to be evaluated in the contemporary wing of the museum. RK&A conducted 100 timing and tracking observations of visitors in the exhibition; visitors were recruited from all three potential entrances to the exhibition. Observation data
Computational Thinking (CT) is a relatively new educational focus and a clear need for learners as a 21st century skill. This proposal tackles this challenging new area for young learners, an area greatly in need of research and learning materials. The Principal Investigators will develop and implement integrated STEM+C museum exhibits and integrate CT in their existing engineering design based PictureSTEM curriculum for K-2 students. They will also pilot assessments of the CT components of the PictureSTEM curriculum. This work will make a unique contribution to the available STEM+C learning materials and assessments. There are few such materials for the kindergarten to second grade (K-2) population they will work with. They will research the effects of the curriculum and the exhibits with a mixed methods approach. First, they will collect observational data and conduct case studies to discover the important elements of an integrated STEM+C experience in both the formal in-school setting with the curriculum and in the informal out-of-school setting with families interacting with the museum exhibits. This work will provide a novel way to understand the important question of how in- and out-of-school experiences contribute to the development of STEM and CT thinking and learning. Finally, they will collect data from all participants to discover the ways that their activities lead to increases in STEM+C knowledge and interest.
The Principal Investigators will build on an integrated STEM curriculum by integrating CT and develop integrated museum exhibits. They base both activities on engineering design implemented through challenge based programming activities. They will research and/or develop assessments of both STEM+C integrated thinking and CT. Their research strategy combines Design Based Research and quantitative assessment of the effectiveness of the materials for learning CT. In the first two years of their study, they will engage in iterations on the design of the curriculum and the exhibits based on observation and case-study data. There will be 16 cases that draw from each grade level and involve data collection for the case student in both schools and museums. They will also use this work to illuminate what integrated STEM+C thinking and learning looks like across formal and informal learning environments. Based in some part on what they discover in this first phase, they will conduct the quantitative assessments with all (or at least most) students participating in the study