Zoo and Aquarium Research Collaborative (ZAARC) is a NSF-funded research project by TERC and Oregon State University (co-PI’s Andee Rubin and John Falk) that is exploring a collaborative model for action research as a form of professional development for zoo and aquarium educators. Over the past three years, ZAARC has engaged education staff from six zoos and aquariums from across the country in learning about and conducting action research pertinent to their institutions. Dr. Cynthia Char of Char Associates, an independent consulting firm specializing in program evaluation, has been a member
The most important consideration in evaluating chemistry outreach efforts is how to best use the evaluation to serve project needs. Evaluation should be about making programs more effective—at communicating ideas, changing attitudes, inspiring action, or reaching wider audiences, for example. A well-conducted evaluation typically contributes to the quality of a project by helping its leaders better define their goals, identify important milestones and indicators of success, and use evidence to support ongoing improvements. At its best, evaluation is an integral part of project design and
The Vaccine Buzz exhibition opened at the Science Museum of Minnesota in February 2014. This summative report describes how visitors experienced and interpret the Vaccine Buzz exhibition as a whole. This approach allowed us to understand what visitors were most interested in, at what level the exhibit elements are connected to each other, as well as how and where the main messages are being connected between exhibit elements.
To answer these questions we conducted three different interconnected studies: 1) a counting study to understand how many visitors interact with the Vaccine Buzz
Maker Corps is a program delivered by the Maker Education Initiative (Maker Ed) to increase organizational capacity to develop and deliver maker programing. Since its inception in 2013, the program has grown to support over 100 organizations by providing professional development, connections to a community of other maker educators and individualized support. Over time the program elements have changed in response to feedback from participants, collaboration with evaluators and shifts in focus for Maker Ed’s goals. In the spirit of maker education – tinkering, observing, responding, iterating –
As part of the National Science Foundation (NSF) funding for the In Defense of Food project directed by Kikim Media, the independent evaluation firm Knight Williams Inc.1 conducted a summative evaluation of the project’s key deliverables, which included: a PBS television broadcast program, an outreach effort, and an educational curriculum. This report (Study 3 of 3) considers the In Defense of Food curriculum and, in particular, educators’ reactions to the curriculum in terms of perceived appeal, ease of implementation, and learning value. Feedback was gathered from educators who were surveyed
As part of the National Science Foundation (NSF) funding for the In Defense of Food project directed by Kikim Media, the independent evaluation firm Knight Williams Inc.1 conducted a summative evaluation of the project’s key deliverables, which included: a PBS television broadcast program, an outreach effort, and an educational curriculum. This report (Study 2 of 3) focuses on the outreach effort.
As part of the National Science Foundation (NSF) funding for the In Defense of Food project directed by Kikim Media, the independent evaluation firm Knight Williams Inc. conducted a summative evaluation of the project’s key deliverables, which included: a PBS television broadcast program, an outreach effort, and an educational curriculum. This report (Study 1 of 3) considers the film’s overall appeal, clarity, learning value, and motivational impact among viewers matching the film’s target audience, and focuses on the following six questions:
1) Did viewers find the film appealing, engaging
Science, Technology, Engineering, and Mathematics (STEM) education and workforce development in the US are critical for global competitiveness and national security. However, the U.S. is facing a decrease in entrants to the STEM workforce which is not shared evenly across demographics. Specifically, women, underrepresented minorities, and people with disabilities obtain STEM degrees and enter the STEM workforce at levels significantly below their demographic representation in the U.S. The National Science Foundation's (NSF) Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES) program supports models, networks, partnerships and research to ensure the broadening participation in STEM of women, members of racial and ethnic groups that have been historically underrepresented, persons of low socio-economic status, and people with disabilities. This conference focuses on collective impact as a strategy to address the broadening participation challenge. Collective impact is distinguished from collaboration in that the alliances require a backbone organization to succeed. The goal of this project is to organize a conference to inform backbone organizations toward broadening participation in STEM education and the workforce.
The conference takes place at the University of California, San Diego January 20-22, 2017 and brings together Project Investigators from the Design and Development pilots, along with stakeholders in broadening participation in STEM on a local, regional, and national scale. The overarching goal of the conference is to develop the knowledge base of participants in the application of the collective impact model, and the role of backbone organizations to address specific issues and transition points of the STEM pipeline. Conference participants include K-12, community college, and university representatives; leaders in graduate education, policy makers and private sector employers. The conference includes plenary sessions, flash presentations, and interactive workgroups engaged in the development of collective impact approaches to problems in Broadening Participation in STEM. Workgroups share their insights, and audience feedback is electronically curated via Twitter and Storify. To respond in real time to participant questions or insights this conference uses the innovative platform, IdeaWave, to solicit, sort and value ideas from the attendees before, during, and after the conference. Conference results are integrated into a final report to inform the NSF INCLUDES Alliances backbone organizations. The intellectual merit of the project is that it advances knowledge about the barriers to broadening participation in STEM education and the workforce, the collective impact model, and the role of the backbone organization to guide the vision and strategy, and support the activities, evaluation, and communication of the NSF INCLUDES Alliances. The broader impact of this project is that it benefits society by informing backbone organizations, which leads to broadening participation of the STEM workforce and ultimately increases U.S. global competitiveness and national security.
DATE:
-
TEAM MEMBERS:
Kim Barrett
resourceprojectProfessional Development, Conferences, and Networks
As the NSF INCLUDES Program seeks to scale from the Launch Pilots to the full program, achieving its goals to promote the progress of science by broadening participation will rely on the ability to successfully scale the technical and human infrastructure for collaboration across the mini-backbones and the national backbone. The American Association for the Advancement of Science (AAAS) is seeking support for an NSF INCLUDES infrastructure conference that will provide a forum for discussion about current and long term technical and human infrastructure needs for scaling. Technical infrastructure might refer to the functions provided by any communication, community building and collaboration tool, such as document sharing or storage. Human infrastructure might refer to data analysts or community managers.
The conference would include discussion of the structures and processes for creating a shared, overarching vision of the changes at all levels and for all groups that would be needed to promote the talent development goals envisioned within INCLUDES, supporting the current design and development launch pilots, and supporting scaling and promoting conditions for sustainability. Based on research on collective impact and improvement science in education, we would offer presentations followed by structured conversations within the setting of working sessions. The goals of AAAS for diversity and inclusion in science, technology, engineering and mathematics (STEM) are very much in keeping with the goals of INCLUDES; thus AAAS proposes to offer its existing online platform, Trellis, to support this comprehensive initiative to develop STEM talent from all sectors and groups in our society.
The goals of the conference activity are to: (1) define short term and long term communications and networking needs that can support the pilots; (2) outline the technical specifications and human resources needed to support the pilots; (3) envision the technical, resource and human needs required to support Alliances; (4) develop design specifications for intra- and cross Alliance networking to support technical assistance, identification and curation of resources, support for communities of practice and capture of lessons learned and (5) propose tools, techniques, capacities and functionalities for an NSF INCLUDES National Network Backbone.
DATE:
-
TEAM MEMBERS:
Shirley MalcomJosh Freeman
resourceprojectProfessional Development, Conferences, and Networks
This award supports a conference and related activities which will bring together researchers, higher education administrators, industry representatives, members of professional societies, and other community members to design collective impact initiatives to improve the retention and continuation of Hispanics, women, and other underrepresented groups in STEM fields. Retention and continuation to the next step in their educational careers is a particular challenge for underrepresented minorities and first-generation students, whether in community college, four-year institutions, or graduate programs (Ishitani 2006, Sowell et al. 2015). The conference focuses explicitly on this critical issue, and will identify potential remedies that can scale across different types of institutions, communities, and underrepresented groups.
This conference will employ a collective impact, networked improvement approach to help fill this gap and engage the broader community. This approach offers a means of identifying and agreeing upon shared, measurable objectives, mechanisms for coordination across individuals and organizations, a commitment to contrast and comparison, and an ethic of continuous improvement. Drawing from the extant literature and recommendations from a wide array of stakeholders, the team will propose measurable strategies for Alliances and the National INCLUDES (Inclusion across the Nation of Communities of Learners that have been Underrepresented for Diversity in Engineering and Science) Network that can be implemented, tested, enhanced, and expanded. The workshop team will draw on evidence-based knowledge, leveraging findings from disciplinary and interdisciplinary fields and from differing types of institutions and educational levels to determine whether strategies identified can yield large-scale progress towards INCLUDES goals. Results will inform the INCLUDES Alliances and the backbone organization supporting the Alliances.
DATE:
-
TEAM MEMBERS:
Marjorie ZatzAnn GatesDeborah Santiago
resourceprojectProfessional Development, Conferences, and Networks
The National Science Foundation's (NSF)Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES) program supports models, networks, partnerships and research to ensure the broadening participation in STEM of women, members of racial and ethnic groups that have been historically underrepresented, persons of low socio-economic status, and people with disabilities.
The University of Akron will convene a two-day conference to develop a backbone organization to support the preparation and advancement of underrepresented minorities K-12 through careers in the biosciences, a high growth area for engineering (biomechanics, biometrics and biomaterials). This conference draws on the expertise of a wide range of organizations, professional associations, K-20+, community based organizations, industry and museums. The intent is to strengthen the network among participants and leverage learning on how to engage youth in the biosciences.
The results of this first conference will be a white paper that will be disseminated to several professional societies that outlines a backbone infrastructure for addressing both short-term and longer-term aspects of an NSF INCLUDES alliance centered on bioengineering, biomechanics, biomedical engineering and biomaterials.
DATE:
-
TEAM MEMBERS:
Brian DavisCarin HelferRouzbeh Amini
resourceprojectProfessional Development, Conferences, and Networks
This project will develop, organize and host a Conference to assist in the planning of the NSF INCLUDES (Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science) Alliances and National Network. The focus of this project is the design and implementation of program evaluations necessary for the INCLUDES Alliances and National Network. The conference will be the first to emphasize the multiple scales at which evaluations occur for programs that include components based at single institutions, at interdisciplinary centers/institutes within larger institutions, at alliances/partnerships between several institutions and at entities with a national scope. As the INCLUDES Alliances and National Network will operate across multiple scales, the proposed Conference will provide guidance on the technical aspects of designing and planning multi-scale evaluations.
The target audience for the activities are those individuals involved in current INCLUDES projects, those considering collaborating in such projects and STEM educators considering inclusion of formal evaluation in their projects. The overall goals are to (i) enhance the knowledge of the participants about evaluation methods; (ii) present the experiences of individuals who have successfully developed alliances and carried out evaluation efforts for these; and (iii) provide advice regarding evaluation methods for those planning to participate in future requests for INCLUDES Alliances and/or the National Network. The methods proposed include three components: (a) an initial webinar open to the broad community of STEM educators to introduce key concepts and vocabulary concerning evaluation methods; (b) a pre-Conference Tutorial with 30 participants to provide a one-day overview of modern methods in evaluation and connect these to projects suggested by the participants; and (c) a two-day Conference with 80 participants including as speakers leading experts on program evaluation as well as leaders of programs in STEM education that have operated at multiple scales. Virtual participation will be integrated throughout the three components of the project and a formal evaluation of the project will be carried out using the Systems Evaluation Methodology that has been in use at the host NSF-sponsored Center (NIMBioS) for many years.
DATE:
-
TEAM MEMBERS:
Louis GrossSuzanne LenhartErnest BrothersPamela Bishop