In concert with the overall strategy of the Advancing Informal STEM Learning (AISL) program to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments, Principal Investigators from Oregon State University, University of Idaho, and University of Texas at Dallas, will study a range of data in online social networks to identify evidence of the long-term impact of informal STEM education. Tracking informal learners over time to understand the impact of informal learning experiences has been a longstanding, daunting, and elusive challenge. Now, with massive amounts of data being shared and stored online, education researchers have an unprecedented opportunity to study such data and apply data analytics and visualization technologies to identify the long-term, cascading effects of informal STEM learning. Research findings will inform the design and development of a data-analysis tool for use by education practitioners to improve STEM learning experiences online, through television and film, and at informal education institutions. An independent external critical review board of learning scientists, computer scientists, engineers, informal STEM education practitioners, participating partners, broadcast media professionals, and policymakers, will ensure a robust evaluation of the research and effectiveness and utility of the data analysis tool to improve practice. A summary report for the field will be written on the scientific and practical reliability and validity of the research and data-analysis model, and the value of the work for audiences beyond informal STEM education practitioners and policymakers. The research is contemporaneously relevant, advancing innovative use of data-mining and data-analysis processes to better understand how informal learners communicate STEM learning experiences and interact with STEM content over time, across a range of social networks. Investigators will research: 1) whether learners who engage in informal STEM education experiences further their learning through discussions and sharing of information in social media networks, 2) which types of data are present in social media that are relevant for understanding the cascading impacts of learning over time, and 3) how learning may evolve independently within shared social networks, which, if discovered, could provide a predictive computational model with implications for significant impact across both formal and informal education. Investigators will employ existing and modified data crawlers to search for key terms and phrases, assess spikes and deformations in posts, queries, and blogs, and experiment with their test data to find which types or configurations of keywords or search terms deliver the most reliable and accurate results. A variety of formats will be explored to test various strategies with participating partners and practitioners. Data will be visualized to represent the following dimensions of learning: a) Interest/Affect, b) Recommendations, c) Understanding/Knowledge-Seeking, and d) Deeper Engagement.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This project will study why (or why not) young career adults, aged 18-35 engage with the PBS NewsHour science content via broadcast and/or online avenues to advance their STEM knowledge and skills. This age group has shifted away from viewing traditional broadcast news media and increasingly looks to social media channels for science content. Multiple layers of STEM digital content delivered across multiple platforms (including social media) will be used to identify the attributes that engage and motivate these 18-35 year olds. Deliverables include 12 broadcast segments each year with STEM research coverage and a range of transmedia efforts (e.g. additional formats distributed via Instagram, Vine, YouTube, etc.) for testing with the target audience. A complementary component of the project will be an apprenticeship program in which each year five college age students from journalism schools join the professional reporters at the NewsHour to produce STEM content using new and innovative strategies engage to 18-35 year olds. The PBS NewsHour broadcast is currently viewed by 1.4 million adults each night and the website has 2.6 million unique visitors each month. The research will attempt to define the learning ecologies of 18-35 year olds using psychographic profiles and case studies to illustrate the range of science learners including those in underrepresented groups. The first research component uses a quantitative approach to assess the reaction of the early career adults to the 12 STEM broadcast segments in their original form and after repackaging for social media. A control group audience will watch the original broadcast of each STEM segment and respond to an online questionnaire that will establish how viewers use and/or pass on STEM content and to whom. The test audience will view the content that has been repackaged and presented on a different media platform responding to the same online questionnaire and allowing comparisons of the two groups. The second research component will focus on the college-age journalism apprentices and use participatory action research. The apprentices will collect data about their experiences and reflect on their contributions to STEM reporting. The third research component will be an ethnographic study of the post-production and editorial teams at the PBS NewsHour using focus groups to elicit feedback and evaluate their metacognitive thinking about how to produce stories for early career adults. Data will be collected and analyzed from three groups: early career adults 18-35 years of age; journalism apprentices; and the PBS NewsHour editorial teams. Overall the research will provide new knowledge about producing and distributing digital STEM media that engages and impacts early career adults.
The goal of "Communicating STEM -- Applying the Science of Science Communication to Natural History Media Products in Development/Production" is to bring standard methodology for media product development/production into better alignment with evidence-based best practices for science communication. Presentations in the professional development science strand at the Jackson Hole Wildlife Film Festival (JHWFF) conference will be curated to demonstrate how adhering to research-based communication strategies has been proven to increase knowledge retention in a lay audience and decrease instances of audience bias. The strand will present emerging methods for assessing media impact beyond simply the number of viewers, as well as in depth case studies examining evidence for measurable benefits to adopting science communication strategies. By establishing an international cohort of interdisciplinary professionals, and by recruiting ongoing engagement through dissemination of project deliverables through partner organizations and university programs, JHWFF will increase opportunities for cross-industry collaboration and provide media producers, STEM professionals and science communication experts with the resources and network necessary for informed, effective public outreach through natural history and science media products. This conference strand is funded by the Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. Jackson Hole Wildlife Film Festival (JHWFF) is a premier international industry conference for natural history media professionals. This project will establish a cross-industry, interdisciplinary professional development science strand in the upcoming 2015 JHWFF conference. The goal is to advance interdisciplinary collaborations between STEM professionals, science communication experts, media producers, and students/early career professionals. The strand will explore examples of successful science/media collaborations, and increase discourse on best practices for public engagement at the intersection of STEM research, empirically-proven communication methods, and media content for diverse audiences. The project is divided into two phases: Phase I involves the work at the conference; Phase II will provide free online access to edited videos of program sessions made broadly available through partner organizations and institutions, and promoted via social networking, cohort groups, and online blogs. The collaborating organizations (American Association for the Advancement of Science, LifeOnTERRA, and Participant Media), complemented by a broad group of expert advisors, will extend the capacity of the project, facilitate access to stakeholders, and recruit broader participation in both phases of the project. Dr. Louis Nadelson, Director of the Center for the School of the Future at Utah State University, will conduct external evaluation.
This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This study will contribute to these goals by providing empirical evidence about how and under what circumstances science museums, science centers, and other designed settings for informal science education (ISE) can change so that they will reach more diverse audiences with the full range of their educational activities. This study will deepen understanding of equity norms and practices at ISE organizations, as well as the internal and contextual factors that shape them. The project builds on a unique opportunity provided by the widely known and critically acclaimed exhibition "RACE: Are We So Different?" After traveling to dozens of museums and science centers, and being seen by over three million people, RACE is returning home to the Science Museum of Minnesota (SMM) where it was created. Strand 1 of this study will examine the processes underlying organizational change at SMM as it attempts to use RACE to leverage organization-wide change. In Strand 2, the project team will compare the experiences of ISE organizations around the country that hosted RACE, focusing on the conditions that influence reflection and lead to (or prevent) lasting impact. These two studies will inform the design of Strand 3: a national survey on equity norms and practices, and the potential for equity-related change, in designed settings for ISE. Four research questions guide all project activities: 1)How can such Informal Science Education organizations leverage an unusual event, such as a traveling exhibition, to catalyze and sustain change in their equity-related norms and practices? 2) How and when does the deep reflection required to change entrenched norms and practices manifest itself in ISE organizations that attempt to change their equity norms and practices? 3) What contextual factors support or oppose the achievement and maintenance of organization-wide changes in equity norms and practices? 4) How common are these supportive and opposing conditions in the institutional field of museums and science centers, and how prevalent are different norms and practices related to equity?
A recent report by the Association for Computing Machinery estimates that by decade's end, half of all STEM jobs in the United States will be in computing. Yet, the participation of women and underrepresented groups in post-secondary computer science programs remains discouragingly and persistently low. One of the most important findings from research in computer science education is the degree to which informal experiences with computers (at many ages and in many settings) shape young people's trajectories through high school and into undergraduate degree programs. Just as early language and mathematics literacy begins at home and is reinforced throughout childhood through a variety of experiences both in school and out, for reasons of diversity and competency, formal experiences with computational literacy alone are insufficient for developing the next generation of scientists, engineers, and citizens. Thus, this CAREER program of research seeks to contribute to a conceptual and design framework to rethink computational literacy in informal environments in an effort to engage a broad and diverse audience. It builds on the concept of cultural forms to understand existing computational literacy practices across a variety of learning settings and to contribute innovative technology designs. As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds new approaches to and evidence-based understanding of the design and development of STEM learning in these settings. This CAREER program of research seeks to understand the role of cultural forms in informal computational learning experiences and to develop a theoretically grounded approach for designing such experiences for youth. This work starts from the premise that new forms of computational literacy will be born from existing cultural forms of literacy and numeracy (i.e., for mathematical literacy there are forms like counting songs -- "10 little ducks went out to play"). Many of these forms play out in homes between parents and children, in schools between teachers and students, and in all sorts of other place between friends and siblings. This program of study is a three-phased design and development effort focused on key research questions that include understanding (1) how cultural forms can help shape audience experiences in informal learning environments; (2) how different cultural forms interact with youth's identity-related needs and motivations; and (3) how new types of computational literacy experiences based on these forms can be created. Each phase includes inductive research that attempts to understand computational literacy as it exists in the world and a design phase guided by concrete learning objectives that address specific aspects of computational literacy. Data collection strategies will include naturalist observation, semi-structured, and in-depth interviews, and learning assessments; outcome measures will center on voluntary engagement, motivation, and persistence around the learning experiences. The contexts for research and design will be museums, homes, and afterschool programs. This research builds on a decade of experience by the PI in designing and studying computational literacy experiences across a range of learning settings including museums, homes, out-of-school programs, and classrooms. Engaging a broad and diverse audience in the future of STEM computing fields is an urgent priority of the US education system, both in schools and beyond. This project would complement substantial existing efforts to promote in-school computational literacy and, if successful, help bring about a more representative, computationally empowered citizenry. The integrated education plan supports the training and mentoring of graduate and undergraduate students in emerging research methods at the intersection of the learning sciences, computer science, and human-computer interaction. This work will also develop publically available learning experiences potentially impacting thousands of youth. These experiences will be available in museums, on the Web, and through App stores.
Despite strong efforts by many people and institutions and a deep, ongoing commitment from the National Science Foundation, progress remains uneven and slower than desired with respect to broadening participation of people from all parts of society in the science, technology, engineering, and mathematics (STEM) fields. The broadening participation challenge will become even more urgent with increasing demographic and socioeconomic changes underway in our nation. Through this conference and workshop grant, the Association of Science-Technology Centers (ASTC) will convene a group of diverse thought leaders from across higher education, profit, non-profit, K-12 and informal STEM education sectors for one day of brainstorming and prioritizing possible ideas, strategies, and actions that could be aggressively pursued by broadening participation initiatives. The findings of this workshop could support ongoing, field-wide discussions about the next generation of projects and efforts to address issues of underrepresentation in STEM. This workshop will build upon a foundation of existing NSF programs and funded projects and will draw upon ongoing efforts by ASTC's Center for Advancement of Informal Science Education (CAISE) to address broadening participation challenges in informal STEM learning environments. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Inclusion Across the Nation of Communities of Learners that Have Been Underrepresented for Diversity in Engineering and Science (INCLUDES) Leadership Workshop will engage up to 55 local and non-local participants from the higher education, profit, non-profit, K-12 and informal STEM education sectors that have been selected for their extensive but varied experiences with efforts to broaden participation in STEM. Before the workshop, participants will prepare for the plenary talks, panel presentations, and breakout session discussions by reading selected literature about effectively scaling innovations, collective impact strategies, catalytic innovations, and other related theory. Specific goals of this one-day workshop are 1) to consider potential scalable high-impact innovations in STEM education to assure success for all people across the nation; and 2) to generate ideas, strategies, and actions that could substantially alter the current landscape and potentially achieve a transformative change for inclusion. ASTC proposes to disseminate the workshop findings to worksop participants, the broader communities to which participants belong, and even the National Science Foundation. A workshop synthesis report and other content generated at the workshop (speaker slides, presentation video, graphic documentation to name a few) will reside at ASTC's informalscience.org website. ASTC proposes an extensive communications media strategy that will draw stakeholder attention to these resources and support field-wide discussion and action around broadening participation.
Youth Volunteer Interpreters as Facilitators of Learning about Climate Change in Zoo Settings reports on a project to pilot test climate change education resources featuring youth volunteer interpreters as facilitators for zoo visitors’ experiences. Brookfield Zoo tested inquiry-based and specimen-based interpretation at the polar bear and Humboldt penguin exhibits, and Woodland Park Zoo tested a climate-change themed "activity cart" near their Sumatran tiger exhibit. Both the effects of youth volunteer interpretation on the zoo visitors and on the youth volunteer interpreters themselves were
Global Climate Change as Seen by Latin American Zoo Visitors reports on the findings of a summer 2011 survey conducted at eight Latin American zoos in five countries. The study was designed to characterize the readiness of Latin American zoo visitors to engage with the issue of global climate change. This included describing visitors’ cognitive, attitudinal, and behavioral predispositions toward climate change in addition to describing their attitudes and beliefs regarding wildlife, nature, and conservation actions. Results indicate that Latin American zoo visitors have a high degree of
Global Climate Change as Seen by Zoo and Aquarium Visitors reports on the findings of a summer 2011 survey conducted at 15 zoos and aquariums. The study was designed to characterize the readiness of U.S. zoo and aquarium visitors to engage with the issue of global climate change. This included describing visitors' cognitive, attitudinal, and behavioral predispositions toward climate change in addition to describing their attitudes and beliefs regarding wildlife, nature, and conservation actions. Results indicate that zoo and aquarium visitors are receptive audiences for climate change
As Kathleen Enright rightly stated in the opening article of this issue: “Until all participants in the evaluation chain embrace and support a learning focus, I think our ability to use evaluation results to increase impact will be limited.” As grantmakers, we arguably play the most important role in ensuring that all stakeholders receive the support they need to be full participants, contributors, learners and beneficiaries of an evaluation process. For evaluation to be worth the money and time we spend on it, everyone needs to be a full participant — funders, grantee organizations and even
The Massachusetts Linking Experiences and Pathways Follow-on (M-LEAP2) is a three-year longitudinal empirical research study that is examining prospectively how early formal and informal STEM education experiences are related to gender-based differences in STEM achievement-related choices in middle and high school. M-LEAP2 serves as a complement to - and extension of - a prior NSF-funded study, M-LEAP, which was a largely quantitative research study that followed overlapping cohorts of 3rd - 6th grade female and male students for three years. M-LEAP surveyed over 1,600 students, 627 student-parent pairs, and 134 second parents in 8 diverse public schools across Massachusetts. In contrast, M-LEAP2 is a heavily qualitative three-year study using in-depth interviews with a diverse range of 72 of these students and their families to study how formal and informal science experiences shape the students' science-related beliefs, interests, and aspirations as they progress though middle and high school.
This annotated bibliography provides selected references regarding multiculturalism and cultural competence in evaluation. It contains a section listing more recent publications dating beyond 2007.