Hacking Your Mind Research Study Report

On October 1, 2015, Oregon Public Broadcasting (OPB) was awarded a 3‑year grant of 2.7 million dollars from the National Science Foundation (NSF) to fund the project Hacking Your Mind (award number 1515520). A major public and social media project, Hacking Your Mind (HYM) planned to engage Americans with the new discoveries being made in the social, behavioral, and economic sciences and the remarkable insights these discoveries offer into how individuals make numerous daily decisions and judgments, as well as the broader impact of this highly personal phenomenon on nearly every aspect of people’s lives. OPB planned to provide three primary products:

• A riveting, 4‑part primetime public television special featuring top names in the social, behavioral, and economic sciences.
• A social media campaign to attract new and younger audiences.
• An experimental research study that would increase understanding of how participants learn in different informal settings and how learning varies by participant type.

HYM aimed to make two overall contributions to the field: (a) broadening access to and engagement in science, technology, engineering, and mathematics (STEM) learning, and (b) enhancing learning in informal STEM. Informal STEM learning environments and experiences provide a unique opportunity to spark interest and engagement in STEM that could lead to STEM careers. According to a 2017 report from the U.S. Bureau of Labor Statistics, there were nearly 8.6 million STEM jobs in 2015. Both women and certain racial and ethnic groups are underrepresented in STEM careers. A 2017 NSF report identified three underrepresented racial and ethnic groups in science and engineering specifically: Blacks, Hispanics, and American Indians or Alaska Natives; other sources include Pacific Islanders. HYM plans to publish the results of this research study, which will provide critical information to informal science educators seeking to attract a broader audience to STEM, and help public media producers of informal science content — and the organizations that promote and disseminate the content — better understand which approaches are most effective. The social media campaign was launched in August 2020 and the 4‑part primetime public television special premiered on September 9, 2020.

The Portland office of RMC Research was contracted by OPB to conduct the accountability evaluation, the formative evaluation, and the experimental research study. RMC Research conducted an experimental research study in spring 2020. Participants were randomly assigned to three study conditions to address five research questions. Each study group completed a preassessment, watched their assigned video(s), and completed a postassessment. There were three study groups:

• The first study group was a high-dose treatment group who watched the Overarching HYM clip and Episode 4 of HYM. This study condition reflects someone watching a teaser and then a full television show. This study group watched approximately 1 hour and 3 minutes of video.
• The second study group was a medium-dose treatment group who watched the Overarching HYM clip and three clips from Episode 4 with more specific examples of the HYM series content—"How Can We Save the Planet?,” “Can Governments Hack Your Mind?,” and “Can Cops Overcome Their Biases?”. This study condition reflected someone watching a series of clips on Facebook. This study group watched approximately 14 minutes of video.
• The third study group was a low-dose treatment group who only watched the Overarching HYM clip. This study condition reflected someone watching one clip on Facebook. This study group watched approximately eight minutes of video.

Research Participants

The primary target audience for the HYM television series is the Public Broadcasting System (PBS) primetime television viewing audience. According to the PBS Research Audience Insight 2016 Annual Report,[4] the PBS primetime audience is older (median age of 65) and college educated (43% have a 4‑year degree). To attract and engage a broader and younger audience, this research study focused on young adults aged 18–25 (i.e., older Generation Z and younger millennial participants). There were 128 participants in this study. The average age of the study participant was 24, 51% were female, 31% were a racial group that is underrepresented in STEM, and 75% self-identified as a science person. Participants represented 22 states across the country.

Findings

The hypotheses were not supported, but the analyses revealed thought-provoking findings. All three study groups (low dose, medium dose, and high dose) experienced content knowledge gains from pre‑to‑post, and the medium and high dose experienced significant gains. This is an unexpected positive finding because regardless of the length of time watching HYM, content knowledge increased. This finding is important to the informal science learning field, as these results suggest that shorter clips may be as effective as full episodes in increasing participants’ content knowledge.

HYM did not have an impact on participants’ career interest in science nor their view of the relevance and importance of science; there was no change pre- to-post. While their overarching views of science did not change, all three study groups reported on the postassessment a strong interest in learning more about the field of decision science and the specific HYM topics.

All participants were engaged in the activity (watching HYM videos and clips) regardless of the study group. Contrary to our hypotheses, these results suggest that a young audience may be just as engaged in the full episode as they are when viewing the clips. This finding is important for OPB because it suggests that OPB could effectively engage a younger audience with either the shorter video clips or the longer format.

After watching the videos participants were asked if they plan to reflect on how to make decisions, discuss any of the topics with others, search for more information about one of the interviewed scientists, or search for more information about any of the topics. The low‑dose participants were significantly less likely to indicate these actions as a result of participating. These findings suggest that although shorter content may deliver knowledge and engage audiences, longer exposure (i.e., at least 20 minutes) may make people more likely to take action.

There were key differences in the findings by subgroups. For example, non‑science people showed significantly greater gains in interest in a career in science, attitudes about the relevance of science, and attitudes about the importance of scientific research than science people. Although HYM did not have an impact on these variables for all participants, the findings suggest that videos such as these may be effective at broadening interest and access to STEM for individuals who do not identify as science people.

In addition to having an impact on non-science people, HYM had a positive impact on science people. Science people scored higher on the content knowledge items on the preassessment and experienced greater gains in knowledge than the non‑science people. Science people were more likely to demonstrate anticipated actions, expressed greater interest in learning more about decision science and HYM topics, and reported higher levels of engagement than non‑science people. These findings have important implications for future research in informal science learning; in addition to an aggregate level analyses, the results should be analyzed by this type of subgroup variable.

Since women and certain racial and ethnic groups are underrepresented in STEM careers, the findings were analyzed by these subgroups. The findings indicate that HYM had a similar impact on females and males. An important distinction is that after watching the videos, females showed more interest in pursuing a career in social, behavioral, and economic sciences than males. HYM had a similar impact for people who are underrepresented in STEM and those who are not. One key difference was that participants underrepresented (UR) in STEM scored lower than non‑underrepresented (non-UR) subjects on the pretest, but they experienced larger gains pre-to-post. These findings suggest that projects such as HYM could be an effective method for attracting a broader audience to STEM.

If you would like to edit a resource, please email us to submit your request.