Goodman Research Group, Inc. conducted an external evaluation of WGBH's NOVA scienceNOW (NsN) multimedia project as part of the project’s award from the Advancing Informal STEM Learning division of the National Science Foundation. The evaluation assessed NsN’s effectiveness in meeting its broad goals for its public and professional audiences, including increasing public appreciation for, understanding of, and engagement in or pursuit of science, increasing science cafe organizers’ and speakers’ perceived expertise and skills, and fostering a community of practice among cafe organizers. The NsN
The Lemelson Center for the Study of Invention and Innovation at the Smithsonian’s National Museum of American History (NMAH) contracted Randi Korn & Associates, Inc. (RK&A) to conduct a formative evaluation for Places of Invention, an exhibition funded by the National Science Foundation. The exhibition aims to stimulate visitors’ thought about how people, resources, and spaces work together to support invention in historic and modern communities. Through formative evaluation, RK&A explores visitors’ use of exhibition prototypes (including barriers to use) and the meanings visitors take away
DATE:
TEAM MEMBERS:
Randi Korn & Associates, Inc.Lemelson Center for the Study of Invention and InnovationNational Museum of American HistorySmithsonian Institution
The C2C award addresses the lack of validated instruments to measure teamwork and collaboration in middle and high school students in out of school time (OST) settings by implementing a rigorous four-phase process to develop new assessments. Phase 1 focuses on defining the construct of teamwork and collaboration skills so it aligns with the research literature and is relevant to outcomes in a variety of STEM OST programs. Construct maps are developed during Phase 2 to guide item development. The instruments are piloted in Phase 3 through think-aloud interviews and survey administration with a diverse set of youth and programs. Through an iterative process, items are revised or removed based on their psychometric properties. The final phase is a national field test with a cross-section of STEM OST programs. C2C's intellectual merit is its potential to advance understanding of how to measure teamwork and collaboration skills in STEM OST programs. There is a national call for more measures to evaluate 21st century skills. C2C's creation of instruments to measure teamwork and collaboration skills in STEM OST programs helps to address this gap. The work of C2C addresses broader impacts and benefit society by creating tools to understand the role STEM OST programs play in readying our nation's youth for the STEM workforce. C2C will create instruments validated specifically for this diverse population, allowing programs to understand the role they play in important societal STEM workforce readiness outcomes. C2C also benefits the informal science education field by conceptualizing the construct of teamwork and collaboration within STEM OST programs and developing validated instruments to understand the impact of these programs on youth.
The Exploring Engagement and Science Identity Through Participation is a research project that examines the outcomes of various citizen science models for Public Participation in Scientific Research (PPSR). The investigation specifically targets citizen science projects that focus on relationships between science learning and science identity. A mixed-method, meta-analysis approach will be employed. The two primary goals of the study are to better understand science learning and science identify within PPSR contexts and to develop a valid Participant Engagement Metric (PEM) for use within PPSR contexts. Data will be collected and analyzed from several PPSR projects and over 4,000 PPSR participants. The project will address several research questions: (1) What are the dimensions of PPSR engagement and how can we measure them across different project models? The research for this question will include approximately 50 interviews of participants in six primary projects representing a range of PPSR approaches to develop and validate a Participant Engagement Metric (PEM) that will be constructed from the frequency, duration and intensity of involvement in key activities. The project will use a constant comparative method of data analysis (processing of data as they are gathered in order to compare them with emerging categories. The development of the PEM will be one concrete outcome of the proposed research. (2) Within and among projects, what is the relationship between participant engagement and individual learning outcomes? The project will use validated measures of learning outcomes derived from Learning Science in Informal Environments (NRC, 2009) and developed for the DEVISE project described below. The six measures include: Behavior & Stewardship, Skills of Science Inquiry, Knowledge of the Nature of Science, Interest in Science & the Environment, Efficacy, and Motivation. The proposed research will look at how engagement affects learning outcomes, as assessed by these six measures (science interest, efficacy, etc.). The project will conduct participant surveys in the six participating citizen science projects in addition to several other PPSR projects to test the relationship of the PEM and other measures of activity involvement to the learning outcomes measured by the survey instruments developed and validated through a previous project. The data analysis will begin with univariate summary statistics that will provide an overview of the basic dataset. Inferential statistics including multiple regression analysis will be used to test the relationships among the independent variables (type of PPSR project, demographics), PEM as a mediator variable, and the dependent variables. (3) How does degree and quality of participation in scientific research develop and/or reinforce individual science identity? The project will conduct a 3-year longitudinal study using surveys, a series of in-depth interviews, and on-site observations of 36 participants from the six primary projects to understand the development of individual science identity within PPSR environments over time. Quantitative and qualitative data and findings will be "triangulated" to determine if convergent, inconsistent or complementary results can be identified. The project proposes a strong dissemination plan, using these approaches, to present research in journals, disseminate research briefs, host online forums, and launching various listserves and online forums at the citizenscience.org website.
Indianapolis / City as Living Laboratory (I/CaLL) is a city-wide civic collaboration engaging in cross-sector research that builds on prior research in informal science learning in public settings. It extends research in place-based and service learning traditions, and uses the city itself as an informal science learning (ISL) environment for Science and Engineering for Environmental Sustainability learning outcomes. This project is creating place-based science learning experiences as part of public spaces in Indianapolis and establishes the next generation of urban science museums that increase opportunities for learning. The project will develop a self-sustaining program for art/science collaborations as they inaugurate city-sanctioned changeable installations at I/CaLL sites. Data from the project will be used to inform ISL professionals at museums throughout the community and around the country. Thousands of volunteers and their families will help create I/CaLL spaces, engage with communities, and serve as research participants connecting with science learning through site development. The unprecedented scale of this project provides a full measure of informal science service learning at a city scale, offering data that can change how science learning is measured, how people from all walks of life develop science literacy as part of their social public experience, and embodying the concept of the city as a living science learning lab. Broader impacts include the development of the city as an informal science learning environment that will become a new standard for thinking about what cities as cultural units can become as we build a resilient Science and Engineering culture for Environmental Sustainability.
National Parks are full of interesting and unusual STEM features which often intrigue visitors whose questions are answered by park personnel. In addition to the natural features, there are often researchers in the parks gathering data and conducting experiments. Park personnel are not apprised of these studies yet are often questioned about them. This collaborative project's goals are to derive a mechanism to educate the park personnel so they can respond to the visitor's inquiries. Collaborators include the National Park Service (NPS), TERC, Winston-Salem State University, and the park personnel at Carlsbad Caverns National Park. The plan is to work through the park interpreters who are employees of NPS and often are the voice for explaining the park's natural features. For this project, the interpreters and researchers will collaborate on the explanations of the science and TERC will work with the interpreters on interacting or educating the public visitors on the research. This is a pilot study to determine how best to bridge the scientists and their research to the park visitors. Evaluation on all elements in this study will be done by Char Associates and the Institute for Mathematics and Science Education at New Mexico State University. The results of this study are to determine the issues in explaining the research to the park interpreters and thence to the park visitors. If successful, it is anticipated that a model will be developed in collaboration with the NPS for use in other National Parks.
Our goal is to demonstrate an educational model fully commensurate with the demands of the 21st Century workforce, and more specifically, with the emerging “green-tech” economy. We recognize a pressing need creating more sustainable solutions for the (human) built-environment and of stabilizing economic patterns that uphold sustainable systems. to prepare citizens for the challenges of The ASCEND model is designed to encourage these societal shifts, but at the same time, it is an attempt to put theory into practice - activating educational practices aligned with research on human development and cognition. For some time now strong recommendations for apprenticeship learning have emphasized the function of legitimate peripheral participation – the possibility of which becomes more prevalent in robust communities of practice. As compared to top-down approaches (typical of formal education settings) these "learning communities" are seen as being more closely aligned with our natural propensities for learning and cognition. ASCEND represents a design-experiment -an attempt to learn how we can create and sustain opportunities for apprenticeship learning in an interdisciplinary arena at the leading edge of technical innovation. In addition, the ASCEND model introduces and examines the efficacy of “digital storytelling” as an alternative to more traditional forms of apprenticeship learning and as a means to engage and advance this and future generations in STEM. A further goal is to develop innovative measures of assessment commensurate with this new model of apprenticeship learning. Finally ASCEND explore how informal learning organizations (museums, libraries, preserves etc.) can use digital storytelling to develop community-driven programs inclusive of at-risk youth and other hard to reach audiences.
Living Liquid is a full-scale development project that will develop and research a new genre of science exhibit that engage visitors in inquiry with large scientific datasets through interactive visualizations. Building on findings from a prior pathways project, Living Liquid will develop three interactive visualizations on a multi-touch Viz Table with a tangible user interface. Each visualization will support visitors in the exploration of a dataset provided by the project’s science partners: 1) Plankton Patterns will show how the ocean is defined by regions of microscopic life using data from the MIT Darwin Project; 2) Ocean Tracks will reveal the “highways” large marine creatures travel with data from the TOPP project at Stanford University; and 3) Genetic Rhythms will follow the activity of marine creatures’ genes in response to environmental conditions based on data from the Center for Microbial Oceanography Research and Education (C-MORE). Through an iterative process of collaborative research and development among museum professionals, educational researchers, computer scientists, marine biologists, data artists and interaction designers, this project seeks to: (1) Advance public understanding of ocean ecosystems and large data inquiry skills through the development of a Viz Table. (2) Advance STEM professionals’ knowledge of how to engage the public in inquiry with visualizations through an educational research study. (3) Increase the capacity of STEM professionals (both ISE developers and research scientists) to develop visualizations through a collaborative development process that includes graduate student training and residencies.
Brigham Young University and the University of Maryland, in partnership with the Smithsonian Institution, the Computer History Museum, and NASA, plus leading game designers, educators, scientists, and researchers, will conduct research on the design and development of two large-scale Alternate Reality Games (ARGs) based on deep-time science in astrobiology, astrophysics, and interplanetary space travel. The project will iteratively design and test two distinct types of ARGs (closed- and open-ended) to study the effects of these ARGs on STEM learning. The ARGs will be based upon the Next Generation Science Standards (NGSS), affording learners with intensive, self-driven, and scaffolded scientific learning and will be aimed at attracting girls and other groups historically underrepresented in science and technology. Each ARG will be designed by NASA scientists, educators and education researchers, and game-based learning experts and will be highly interactive: engaging learners in collaborative investigations in real and virtual worlds to collect scientific data, conduct data analysis, and contribute scientific evidence that will help solve scientific questions within a science-based narrative derived from real world problems that will develop learners' computational thinking skills in a collaborative, participatory virtual learning environment. Combining data from web and social media analytics, player interviews, surveys, and user-generated content, researchers, and evaluation experts at UXR who will provide an outcomes-based evaluation, including front-end, formative, remedial, and summative evaluations, will establish the properties of ARGs that most effectively advance informal STEM learning outcomes. By comparing open-ended and closed-ended ARGs, the PIs will be able to assess the relative strengths and weaknesses of two distinct approaches to Alternate Reality Game design. The project team will test the hypothesis that open-ended, user-generated content will support inquiry-based learning, peer-to-peer learning, and life-wide and life-deep learning, while close-ended, narrative-rich ARGs will support specific transfer of STEM knowledge, collaboration, and problem solving. To help ensure that the games appeal to their target audiences, the project team will adopt co-design methods, enlisting the creative input of participating teens at each stage of the design process. Supplementary materials and lesson plans developed in close consultation with teachers, librarians, teens, and external stakeholders will enable the ARGs to be widely and effectively used as a model in museums, classrooms, libraries, and after-school programs. The proposed ARGs represent a unique environment to test learning principles that enable players to bridge their learning through transmedia across multiple contexts and test the effects of collaboration with massive numbers of concurrent players. As a result, the project should yield insights on how learning principles can be adopted and re-appropriated for emerging learning environments, including those that that might be crowd-sourced. The research is well grounded in the literature and the PIs do an excellent job of mapping ARG design principles to the pertinent learning science research, providing a clear sense of the particular affordances of the genre that should lead to new understandings. The approach has profound implications for the way we might teach the next generation of students. The ability to mix problem solving and learning in virtual spaces with experiences and data derived from the physical world could dramatically change how we understand the role of technology in education.
This three-year research project will study the impact of science center staff facilitation strategies in the area of mathematics learning in a museum exhibit environment. The three main deliverables are: (1) Iteratively developing and refining a theoretical model of how staff facilitation can deepen and extend family mathematical discourse at interactive exhibits; (2) Rigorously testing key components of this model, including the relationship between staff facilitation and the nature of family mathematical discourse; and (3) Providing evidence and research-based tools to support PD efforts for informal STEM educators. The project will leverage the success of the NSF-funded Access Algebra project (DRL-0714634) to advance the field's understanding of socially mediated, informal math learning and identify effective, evidence-based facilitation approaches. The project's research will build from theoretical notion of sociomathematical norms (Yackel & Cobb, 1996), which is currently based on classroom research. A key element of the project will be to determine whether and how, the norms can be applied to informal learning environments. The first phase of the project begins with a qualitative, design-based research (DBR) study to develop a theoretical model of staff-facilitated family math learning, including staff facilitation strategies that support family mathematical discourse and contextual factors that influence that discourse. In the second phase of the project, the team will use an experimental approach to rigorously test the staff facilitation model developed during Phase 1. This mixed-method design will allow the team to both study the complexities of informal math learning and rigorously test causal connections between staff facilitation and the level of family math discourse. Finally, the project staff will provide tools to support PD efforts for informal STEM educators across the country.
This Pathways project from the Ocean Discovery Institute (ODI) seeks to develop and pilot a program model designed to fill an identified gap in citizen science research and practice literature: how to effectively engage and better understand how to foster participation among people from under-represented groups in citizen science research. The ODI model is designed around six principles: (1) leaders who are reflective of the community, (2) science that is locally relevant, (3) guided, as opposed to self-guided, experiences, (4) direct interactions with scientists, (5) progressively increasing responsibilities for participants who express interest, and (6) removing barriers to participation, such as transportation, language, family involvement and access to technology. The project addresses environmentally degraded, crime-ridden local canyons, a locally relevant STEM-related issue, and leverages the Southern California Coastal Water Research Project's (SCCWRP) regional citizen science effort focused on identifying the sources and pathways of trash through regional watersheds. The scientific research components of the project focus on four canyons in the area, employing sampling methods developed by SCCWRP. Youth who are part of other ODI programs and who have demonstrated leadership and interest in science, work with the project team to scaffold family and youth participation in project activities taking place during afterschool and weekend time. Based on continued participation in the project, community participants can become more involved in the project, starting as "new scientists" and moving through "returning scientists" to "expert scientists" roles. The project evaluation seeks to identify the role and importance of the components of the proposed model with respect to participation, retention, and learning by participants from groups under-represented in STEM. The dissemination products of this Pathways project include a white paper describing the model and lessons learned as well as presentations to community groups and education and citizen science practitioners. Based on insights from the iterative approach to the model during this Pathways study, a subsequent full-scale development project would seek to engage citizen science projects around the nation in adapting the model to increase participation of individuals from groups underrepresented in STEM, including building out ODI's citizen science programming.
DATE:
-
TEAM MEMBERS:
Lindsay GoodwinRoxanne RuzicTheresa Sinicrope Talley
The University of Texas at El Paso will conduct a research project that implements and documents the impact of co-generative dialogues on youth learning and youth-scientist interactions as part of a STEM research program (i.e., Work with A Scientist Program). Co-generative dialogues seek to specifically assist with communication and understanding among collaborators. Over four years, 108 11th grade youth from a predominantly (90%) Hispanic high school will conduct STEM research with twelve scientists/engineers (e.g., chemist, civil engineer, geologist, biologist) and undergraduate/graduate students as part of 7 month-long after school program, including bi-weekly Saturday activities for 5 months followed by an intensive month-long, self-directed research project in the summer. Youth will be randomly assigned to experimental groups that include the co-generative dialogue treatment and control groups without the intervention. The scientists and their STEM undergraduate/graduate students will participate in both experimental and control groups, with different youth. Youth will receive high school credit to encourage participation and retention. The PI team hypothesizes that co-generative dialogues will result in improved learning, communication, and research experiences for both youth and scientists. Educational researchers will conduct co-generative dialogues, observations, interviews, and surveys using validated instruments to address the following research goals: (1) To investigate the impact of the treatment (co-generative dialogues) on youth knowledge, attitudes, perceptions of their experience, and their relationships with the scientists; (2) To investigate the impact of the treatment on scientists and graduate students; and (3) To identify critical components of the treatment that affect youth-scientist interactions. It is anticipated that, in addition to providing in-depth STEM research experiences for 108 youth from underrepresented groups at a critical time in their lives, the project will result in widely applicable understandings of how pedagogical approaches affect both youth learning and scientist experiences. The project also seeks to bridge learning environments: informal, formal, university and digital.