The Exploratorium and the Museum of Life and Science will develop, evaluate and implement Science of Sharing, a three-year full-scale development project designed to bring the scientific study of human social behavior to a broad public audience. Science of Sharing will create new ways for visitors to experiment with social psychology and will generate important information for informal science institutions committed to involving visitors in discussions of personal, societal, and scientific responses to real-world challenges. Science of Sharing addresses a critical ISE issue: creating ways for visitors to experiment with inquiry based exhibits and activities that heighten public knowledge of the study of human social behavior. Based on research in social psychology and game theory, the project (a) fosters public engagement in activities exploring collaborative behavior and resource sharing; (b) promotes awareness of connections between these experiences and STEM-related research in psychology and economics; and (c) links individual behaviors to real-world issues of resource depletion and group conflict. The primary audience is youth and youth-adult museum visitors, with particular focus on underrepresented communities with limited access to communication technologies. The secondary audience is ISE professionals with interest in new kinds of interactive experience and visualization tools focusing on social behavior and techniques for fostering social interaction and public discussion of science. The project will (1) conduct front end evaluation to assess visitor attitudes and knowledge about issues of cooperation and resource use; (2) design, prototype, and evaluate 15 inquiry-based exhibits and 4 Experimonths (public events with web, museum, and community-based components on social-psychological topics); (3) conduct design-based research to investigate aspects of these exhibits and activities that prompt self-reflection and build metacognitive skills; and (4) work with local school districts to adapt exhibits for classroom use.
The City as Learning Lab (CaLL) is a comprehensive research and development initiative designed to create new measures of audience impact in technology experiences; identify features of university-community collaboration that facilitate sustainable community programs; and produce a set of tools and resources that allow other cities to tailor creative robotics programs to unique audiences. Project partners include the University of Pittsburgh Center for Learning in Out-of-School Environments (UPCLOSE), the Community Robotics Education and Technology Empowerment (CREATE) lab at Carnegie Mellon University's Robotics Institute, and the Georgia Institute of Technology as well as local museums, community organizations, and afterschool clubs. CaLL builds on the work of three existing youth technology programs in Pittsburgh targeting audiences ages 9-15: the Robot Diaries, Neighborhood Nets, and Robot 250. Research questions relate to creative processes in informal learning settings, use of robotics to engage diverse audiences, and changes in technological fluency after students leave the informal learning setting and apply their new knowledge and skills at home or in other learning contexts. The research incorporates data from up to 1000 program participants. Findings will establish evidence for how technological fluency can be measured, supported, and developed through informal technology learning experiences. Project deliverables include a CaLL curriculum, toolkit, new measures of audience impact, and identification of factors that support university/community collaborations. Broader impacts in informal technology education will be achieved by developing flexible toolkits that allow other communities to adapt and adopt CaLL technologies, curricula, and activities.
The project's goal is to facilitate the growth and use of the web site informalscience.org for posting reports of research and evaluation of Informal Science Education (ISE) funded projects. The project leaders will also synthesize the posted evaluation reports of informal science education research and development projects into readable documents. This synthesis will cover all available data from evaluation and research studies reported to informalscience.org across all sectors of ISE (e.g., museums, after school programs, video, radio, film, and technology). The investigators will provide the ISE community with information about these studies through publication on the site, through peer-reviewed publications for a research and evaluation audience, and through communications at conferences focused on ISE policy-makers and ISE practitioners. The report writing will be managed by a small staff of professional researchers and practitioners at the University of Pittsburgh, Minnesota Museum of Science, and Visitors Studies Association. The project will be continually evaluated by a board of advisors that will provide a yearly written report about the web site and synthesis work. The evaluators are researchers familiar with syntheses and meta analysis methods, experts from media, museum, and community programs, and also experts on development and use of professional development web sites. The evaluation will address whether or not the syntheses of evaluation reports was as rigorous as possible given the type of reports that are available. The usefulness of the reports and web site to the field of practice and research will also be a matter for concern by the advisory board. The long term aim of the project is to create a network that promotes deeper connections between evaluation and practice. Also, the network is expected to meet the needs and working styles of the various ISE sectors and to create exchanges and synergies among them. The site is expected to become more usable and useful to the field in each succeeding year, and it is expected to maximize its impact for practitioners, evaluators, policy makers, and funders.
This research agenda is a living document, constructed in response to on-going field-wide conversations following the 21st Century Natural History Settings Conference at the Smithsonian Museum of Natural History. At the conference, natural history professionals explored new directions for museums and other natural history institutions, including zoos, aquaria, botanical gardens, and nature centers. The research agenda is intended to be edited, discussed, and fleshed out by the field as we work together and make progress. New research questions will emerge spurred by surprising findings
Making Connections, a three-year design-based research study conducted by the Science Museum of Minnesota in partnership with Twin Cities' communities, is developing and studying new ways to engage a broader audience in meaningful Maker experiences. This study draws and builds on existing theoretical frameworks to examine how community engagement techniques can be used to co-design and implement culturally-relevant marketing, activities, and events focused on Making that attract families from underrepresented audiences and ultimately engage them in meaningful informal STEM learning. The research is being done in three phases: Sharing and Listening - co-design with targeted communities; Making Activities Design and Implementation; Final Analysis, Synthesis and Dissemination. The project is also exploring new approaches in museums' cross-institutional practices that can strengthen the quality of their community-engagement. In recent years, Making - a do-it-yourself, grassroots approach to designing and constructing real things through creativity, problem-solving, and tool use - has received increasing attention as a fruitful vehicle for introducing young people to the excitement of science and engineering and to career skills in these fields. Maker Faires attract hundreds and thousands of people to engage in Making activities every year, and the popularity of these events, as well as the number of museums and libraries that are beginning to provide opportunities for the public to regularly engage in these types of activities, are skyrocketing. However, Maker programs tend to draw audiences that are predominantly white, middle class, male, well educated, and strongly interested in science, despite the fact that the practices of Making are as common in more diverse communities. Making Connections has the potential to transform how children begin to cultivate a lifelong interest in engineering at a young age, which may ultimately encourage more young people of color to pursue engineering careers in the future.
The Massachusetts Linking Experiences and Pathways Follow-on (M-LEAP2) is a three-year longitudinal empirical research study that is examining prospectively how early formal and informal STEM education experiences are related to gender-based differences in STEM achievement-related choices in middle and high school. M-LEAP2 serves as a complement to - and extension of - a prior NSF-funded study, M-LEAP, which was a largely quantitative research study that followed overlapping cohorts of 3rd - 6th grade female and male students for three years. M-LEAP surveyed over 1,600 students, 627 student-parent pairs, and 134 second parents in 8 diverse public schools across Massachusetts. In contrast, M-LEAP2 is a heavily qualitative three-year study using in-depth interviews with a diverse range of 72 of these students and their families to study how formal and informal science experiences shape the students' science-related beliefs, interests, and aspirations as they progress though middle and high school.
CDC provides its funded programs with a wide range of evaluation resources and guides. State health departments, tribal organizations, communities, and partners working in a variety of public health areas may also find these tools helpful. The resources provide guidance on evaluation approaches and methods, relevant examples, and additional resources. The guides are intended to aid in skill building on a wide range of evaluation topics. Practical Strategies for Culturally Competent Evaluation is designed to complement the other evaluation resources offered by the Division for Heart Disease and
DATE:
TEAM MEMBERS:
Centers for Disease Control and Prevention (CDC)Derrick GervinRobin KuwaharaRashon LaneSarah GillRefilwe MoetiMaureen Wilce
Educators in informal science are exploring data visualization as a way to involve learners in analyzing and interpreting data. However, designing visualizations of data for learners can be challenging, especially when the visualizations show more than one type of data. The Ainsworth three-part DeFT framework can help practitioners design multiple external representations to support learning.
The goal of the study was to identify current issues for navigating and finding relevant information; evaluate the look and feel of the website in relation to its main purpose; assess the clarity of language to its audience; determine next steps for improvement. We addressed these goals through an expert review based on a 100+ point usability assessment framework on both desktop and mobile as of 12/4/2014 and informal feedback from 3 novice users ( age range 22-27 years).
DATE:
TEAM MEMBERS:
Brigham Young UniversityUniversity of MarylandJes KoepflerNidhi JalwalVictor Yocco
The Dimensions of Success (DoS) observation tool defines and provides rubrics (with levels 1-4) for 12 dimensions that were developed to measure STEM program quality in out-of-school time. This technical report summarizes the development of the instrument and findings from our initial study that included 284 observations in the field across 58 STEM programs in two geographic regions (New England and the Midwest). Data were collected by 46 trained observers who observed in pairs. This report is the initial step in developing a validity argument for the instrument.
DATE:
TEAM MEMBERS:
Ashima Mathur ShahCaroline WylieDrew Gitomer
The Dimensions of Success observation tool, or DoS, pinpoints twelve indicators of STEM program quality in out-of-school time. It was developed and studied with funding from the National Science Foundation (NSF) by the Program in Education, Afterschool and Resiliency (PEAR), along with partners at Educational Testing Service (ETS) and Project Liftoff. In 2014, a technical report was released, describing the tool and its psychometric properties (http://www.pearweb.org/research/pdfs/DoSTechReport_092314_final.pdf). The DoS observation tool focuses on understanding the quality of a STEM activity in an out-of-school time learning environment and includes an explanation of each dimension and its key indicators, as well as a 4-level rubric with descriptions of increasing quality. Today, over 700 people have been trained to use the DoS tool, and over 12 state networks have adopted DoS to measure the quality of their afterschool STEM programming.
DATE:
-
TEAM MEMBERS:
Program in Education, AfterschoolDr. Ashima ShahDrew Gitomer
Concord Evaluation Group (CEG) conducted an evaluation study to learn about the Future City’s impact on students as well as to discover ways to enhance Future City for future implementation. In addition to exploring the program’s impacts, with this study we also had an opportunity to explore potential differences between students who compete at their Regional competitions only versus students who make it to the National competition. In collaboration with DiscoverE, CEG developed four surveys to collect feedback from students, parents, educators, and engineer mentors. These data collection