The National Academies of Sciences' Board on Chemical Sciences and Technology (BCST) and Board on Science Education (BOSE) are collaborating on a three-year project to develop a framework for effective chemistry communications, outreach, and education in informal settings. The initiative will include a "landscape" study that will synthesize lessons learned from practice along with education and learning science research about chemistry learning and teaching in informal and formal settings. The overall process will define a set of principles for engaging the public with chemistry and embed these principles into a broad framework that chemists and informal science education professionals could use to identify a set of effective strategies for a given audience and a given educational or communication goal. The guidance and tools resulting from this activity, which is a chemistry-specific case study, should be more generally applicable to science and engineering communications, informal education and outreach. Findings are also likely to apply to aspects of formal education.
The United States Botanic Garden (USBG) contracted RK&A to explore visitors’ interest in and understanding of the design and interpretation presented in four conservatory rooms—Rare and Endangered Species, Plant Adaptations, Medicinal Plants, and Garden Primeval. Two rooms were quite small and the other two rooms were larger. Each room was designed to support a theme and included a variety of signage to support the theme (e.g., plant ID signs, ephemeral signs, thematic signs). Findings from this study will be used in part to inform a new interpretive strategy for the Garden. How did we
The United States Botanic Garden (USBG) contracted RK&A to explore visitors’ interest in and understanding of proposed “Context Clues.” A Context Clue is an interpretive strategy that uses images, product samples or other objects, and interpretive text to assist visitors in connecting to the big idea: plants are important to people. How did we approach this study? The overall purpose of the evaluation was to determine how to best use Context Clues to communicate the relationship between plants and people. Therefore, RK&A interviewed 40 walk-in visitors who experienced three Context Clues in
Pushing the Limits: Making Sense of Science (PTL) is an NSF-funded program designed to build the capacity of rural and small libraries to provide programming to enhance public understanding of science and math. PTL provides professional support, technical assistance, specially produced video segments, and funding for library professionals and their local science partners to co-facilitate a series of science café-style public discussions with adult patrons. In Phase I of the PTL project (September 2012-August 2013), 20 rural and small libraries piloted the program. In Phase II (September 2013
Pushing the Limits: Making Sense of Science (PTL) is an NSF-funded program designed to build the capacity of rural and small libraries to enhance public understanding of science and math. The program provides professional support, technical assistance, and funding for librarians and local science partners to co-facilitate a series of science café-style guided public discussions with adult patrons using books and specially produced video segments. External evaluator Goodman Research Group, Inc. conducted the second formative evaluation, focusing on the professional development (PD) for the
Randi Korn & Associates, Inc. (RK&A) was contracted to conduct a formative evaluation of The Conservation Trust of Puerto Rico’s (the Trust) National Science Foundation-funded Citizen Science Program, which recruits and trains local Puerto Ricans to conduct scientific research about the Rio Manati watershed alongside Trust scientists, staff, and interpreters. How did we approach this study? The purpose of this evaluation was to explore participants’ motivations, experiences with program logistics, level of engagement, and understanding of project and activity goals, as well as staff and
This Science Learning+ Planning Project will develop a prototype assessment tool (based on a mobile technology platform) to map STEM learning experiences across different learning ecologies (e.g. science centers, mass media, home environment) and to develop research questions and designs for a Phase 2 Science Learning+ proposal. The tool will focus on the impact of the learning ecologies on knowledge, interest, identity and reasoning rather than emphasize learning in a specific content area. The proposing team will develop and conduct a small scale usability study during the planning period, which will inform what is proposed in the Phase 2 research. A key focus of the planning period will be to identify and develop the theoretical constructs (i.e., outcomes) to be measured by the prototype App. As a starting point, the project will start with four of the six strands identified in Learning Science in Informal Environments (National Research Council, Bell et al., 2009): (1) interest triggered by a STEM experience; (2) understanding scientific knowledge; (3) engaging in scientific reasoning; and (4) identifying with the scientific enterprise. Discussion among the project partners during the planning process will revolve around how these strands should be measured in the Phase 2 research across ecologies. The measurement tool will assess the goal(s) that people set as they engage in STEM learning within each ecology and will measure the individuals' duration and level of engagement. The project will strive to utilize measures that: (1) are nonobtrusive; (2) are embedded in STEM experiences; (3) can be used across ecologies; (4) can be scaled for other ecologies than the ones examined in Phase 2 research; and (5) will be easy to use by researchers and practitioners.
DATE:
-
TEAM MEMBERS:
Bradley MorrisJohn DunloskyGreat Lakes Science CenterUniversity of LimerickIdeaStream (UK)Irish Independent newspaper
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This project, "STEM Learning in Libraries: A National Conference on Needs, Opportunities, and Future Directions," brings together libraries, informal educators and STEM education and research organizations to discuss the role of libraries in STEM out-of-school time (OST) education, share existing programs, define library needs, and develop a research and evaluation agenda. To date, there has not been systematic exploration of the ways that STEM programming occurs in libraries nor of their effectiveness when they do happen. This will be the first conference of its kind and stands to have a high degree of visibility and the potential for broad impact. Principal Investigator Paul Dusenbery, Director of the National Center for Interactive Learning (NCIL) and Executive Director of Space Science Institute, will lead an experienced project team that includes Co-PI Keliann LaConte, Lunar Planetary Institute; Susan Brandehoff, Public Programs Office, American Library Association; and Anne Holland, NCIL. The conference sessions will be organized around four strands: (1) showcasing successful STEM programs and reviewing research and evaluation results on informal STEM learning in public libraries; (2) examining the current needs, barriers, and opportunities of public libraries; (3) elucidating the possible future roles of public libraries in the 21st Century; and (4) identifying promising practices and strategies. Beginning with core members comprised of the project team and organizing committees, the project will create a Leadership Forum for library directors, library science educators, and policy makers, as well as STEM professionals and educators. Conference results will be disseminated through a wide variety of organizational websites: NCIL, ALA, LPI, the conference website, the STAR_Net online community, and CAISE. In 2010, there were nearly 1.6 billion visits to 17,000 public libraries. Library audiences are true reflections of the nation's population - they serve all races, ages, economic backgrounds, and regions of the country. The STEM Learning in Libraries conference will give public libraries, STEM organizations, informal educators, and funders an opportunity to begin a dialogue with implications for profoundly impacting the attitudes of millions of Americans toward STEM topics.
This project is making novel use of familiar technology (smartphones and tablets) to address the immediate and pressing challenge of affordable, ongoing, large-scale museum evaluation, while encouraging museum visitors to engage deeply with museum content. Using a smartphone app, museum visitors pose questions to a 'virtual scientist' called Dr. Discovery (Dr. D). Dr. D provides answers and the chance to complete fun mini-challenges. The questions visitors ask are gathered in a large database. An analytics system analyzes these data and a password-protected website provides continuous, accessible evaluation data to museum staff, helping them make just-in-time tweaks (or longer term changes) to exhibit-related content (such as multimedia, lecture topics, docent training, experience carts, etc.) as current events and visitors' needs and interests change. The intellectual merit of this project is that it is building evaluation capacity among informal educators, advancing the fields of visitor studies, museum evaluation, informal science learning, and situated engagement, and is contributing to the development of novel evaluation techniques in museums. This project has many broader impacts: The Ask Dr. Discovery system is available to any venue that wishes to use or adapt it to their context. By enhancing the visitor experience and improving museum access to data for evaluation and data-driven decision making across the country, Ask Dr. Discovery has both a direct and indirect impact on museums and visitors of all types. This project is also training the next generation of STEM and education innovators by employing a diverse team of undergraduate students.
A growing body of research suggests that many types of institutions and experiences contribute to science and technology learning by the public. These include the formal education system, libraries, science centers, citizen science, after school programs, television programs, film and video, newspapers, radio, books and magazines, the internet, community and health organizations, environmental organizations, and conversations with friends and family. It is also important that research about who learns and how they learn from these institutions and experiences is informed by and discussed within the communities themselves. This project focuses specifically on STEM learning in interactive science centers. It invites science center professionals from around the country to contribute to and share findings from an on-going research study designed to better understand the influence that interactive science centers have on youth and adult's long-term understanding, interest and engagement with science and technology. The pair of workshops supported by this grant will engage science center administrators, educators, exhibition and program designers, evaluators and researchers in two sets of tasks. The first workshop, held prior to the start of the larger research project, will examine and focus the research's overall goals and help frame a set of specific research questions. Although no research study can answer all important issues, the goal of the workshop is to ensure that the planned investigation attempts to address the issues and outcomes considered most critical to the broader science center community. The second workshop will occur after completion of all data collection and initial data analysis. It will engage a broad cross-section of the science center community in discussing findings, brainstormind implications and usage, and developing dissemination strategies to insure that the findings of the research reach the broader science center and policy communities. The goal of these workshops is to use the collective wisdom of dozens of active professionals from across the country to develop a suite of strategies for grounding research in practice, incorporating research results into practice as well as bringing important research findings to a broader national audience.
EvalFest (Evaluation Use, Value, and Learning through Festivals of Science and Technology) will test innovative evaluation methods in science festivals that are being held across the country and assess in what ways and how effectively they are used. Morehead Planetarium and Science Center (at the University of North Carolina-Chapel Hill) and the University of California, San Francisco, in collaboration with over twenty science festivals, will (1) investigate whether a multisite evaluation approach is an effective model for creating common metrics for informal STEM education, (2) develop common methods to measure the effects of Festivals, (3) create a query-able database of 50,000 Festival attendees to share with the informal STEM learning field, and (4) document whether these efforts also result in new knowledge related to informal STEM education. The project will develop the Enterprise Feedback Management (EFM) system and query-able database for the festival community. EFMs are systems, including processes and software, that enable groups (such as the festival network) to collect, organize, analyze and share data. The EFM system will be designed to integrate data across sites and to allow users to extract data of interest. The project will refine evaluation tools currently used within the Science Festival Alliance that assess self-reported festival learning, and the effects of festival attendance, motivation, and future science participation. It will collect economic impact data and longitudinal festival attendee data. The project will also develop some new evaluation tools such as secret shopper observational protocols. Data from festival attendees will be collected onsite at participating festivals.
The Citizen Science Embedded Assessment project will explore the use of embedded assessment to measure participant science inquiry skill development within the context of citizen science projects. Citizen science (CS) projects partner volunteers with scientists to participate directly in research endeavors. Embedded assessments (EAs) assess participant skills and performance that are directly integrated and are indistinguishable from day-to-day activities. As such, EAs allow learners to demonstrate their science competencies through tasks that are integrated seamlessly into the learning experience itself. The CS field has a growing inventory of self-assessment tools, however, the evaluation of citizen science (and other informal science projects) using such subjective assessments can be remarkably improved when these are used in combination with objective measures of knowledge, skills or other resources participants gain through their participation. Science skills, such as data collection and analysis, are particularly important for CS projects because of their focus on the scientific process and their need for rigorous data collection. Despite the focus on skill gains, CS projects rarely measure such improvements. Embedded assessments (EAs) offer a critical method for understanding the impacts of these participatory learning environments. The project will develop and field test EAs on citizen science topics with an environmental science focus. It will also design training to support their use by individual projects. The project has three primary research foci: (1) identifying common and unique science inquiry skills targeted by CS projects, and how skills are currently being measured to document project impact; (2) identifying the opportunities and challenges present in developing and administering EA tools customized for CS projects to assess science inquiry skills; and (3) assessing whether EA tools created for a CS project can provide project leaders with a better understanding of their project's impact on participant science inquiry skills. The project will address these questions with a needs assessment of research and evaluation studies within the CS community and case studies to develop and test EAs customized for three identified and interested CS projects.