EvaluATE is a national resource center dedicated to supporting and improving the evaluation practices of approximately 250 ATE grantees across the country. EvaluATE conducts webinars and workshops, publishes a quarterly newsletter, maintains a website with a digital resource library, develops materials to guide evaluation work, and conducts an annual survey of ATE grantees. EvaluATE's mission is to promote the goals of the ATE program by partnering with projects and centers to strengthen the program's evaluation knowledge base, expand the use of exemplary evaluation practices, and support the continuous improvement of technician education throughout the nation. EvaluATE's goals associated with this proposal are to: (1) Ensure that all ATE Principal Investigators and evaluators know the essential elements of a credible and useful evaluation; (2) Maintain a comprehensive collection of online resources for ATE evaluation; (3) Strengthen and expand the network of ATE evaluation stakeholders; and (4) Gather, synthesize, and disseminate data about the ATE program activities to advance knowledge about ATE/technician education. The Center plans to produce a comprehensive set of evaluation resources to complement other services, engaging several community college-based Principal Investigators and evaluators in that process.
EvaluATE's products are informed by current research on evaluation, the National Science Foundation's priorities for the evaluation of ATE grants, and the needs of ATE PIs and evaluators for sound guidance that is immediately relevant and usable in their contexts. The fundamental nature of EvaluATE's work is geared toward supporting ATE grantees to use evaluation regularly to improve their work and demonstrate their impacts. All of EvaluATE's products are available to the public. EvaluATE's findings from the annual survey of ATE grantees aid in advancing understanding of the status of technician education and illuminate areas for additional research. The new survey investigates ATE grantees' work to serve underrepresented and special populations, including women, people of color, and veterans. Survey data are available upon request for research and evaluation purposes.
DATE:
-
TEAM MEMBERS:
Lori WingateArlen GullicksonEmma PerkKelly RobertsonLyssa Becho
resourceprojectProfessional Development, Conferences, and Networks
Jobs are growing most rapidly in areas that require STEM knowledge, causing business leaders to seek skilled American workers now and in the near future. Increase in the number of students pursuing engineering degrees is taking place but the percentages of underrepresented students in the engineering pipeline remains low. To address the challenge of increasing the participation of underrepresented groups in engineering, the National Society of Black Engineers, the American Indian Science and Engineering Society, the Society of Hispanic Professional Engineers, and the Society of Women Engineers have formed the 50K Coalition, a collaborative of over 40 organizations committed to increasing the number of bachelors degrees awarded to women and minorities from 30,000 annually to 50,000 by 2025, a 66% increase. The 50K Coalition is using the Collective Impact framework to develop an evidence-based approach that drives management decision-making, improvements, sharing of information, and collective action to achieve success. The first convening of the 50K Coalition in April, 2016, brought together 83 leaders of the engineering community representing 13 professional societies with over 700,000 members, deans of engineering, minority engineering and women in engineering administrators from 11 leading colleges of engineering, and corporate partners representing six global industries. Consensus was reached on the following Common Agenda items: 1.) Undergraduate support and retention; 2.) Public awareness and marketing; 3.) K-12 support; 4.) Community College linkages; 5.) Culture and climate. The Coalition will encourage member organizations to develop new programs and scale existing programs to reach the goal.
The Coalition will use shared metrics to track progress: AP® Calculus completion and high school graduation rates; undergraduate freshmen retention rates; community college transfer rates and number of engineering degrees awarded. The 50K Coalition will develop the other elements of the Collective Impact framework: Infrastructure and effective decision-making processes that will become the backbone organization with a focus on data management, communications and dissemination; a system of continuous communication including Basecamp, website, the annual Engineering Scorecard, WebEx hosted meetings and convenings; and mutually reinforcing activities such as programs, courses, seminars, webinars, workshops, promotional campaigns, policy initiatives, and institutional capacity building efforts. The National Academy of Sciences study, Expanding Underrepresented Minority Participation: America's Science and Technology Talent at the Crossroads recommended that professional associations make recruitment and retention of underrepresented groups an organizational goal and implement programs designed to reach that goal by working with their membership, academic institutions and funding agencies on new initiatives. While these types of organizations work together now in a variety of ways, the relationships are one-on-one. The 50K Coalition brings together, for the first time professional societies, engineering schools, and industry to consider what mutually reinforcing activities can most effectively encourage students from underrepresented groups to complete calculus and graduate from 4-year engineering programs.
DATE:
-
TEAM MEMBERS:
Karl ReidBarry CorderoSarah EcohawkKaren Horting
The Morgan State University INCLUDES project will build on an existing regional partnership of four Historically Black Colleges and Universities that are working together to improve STEM outcomes for middle school minority male students that are local to Morgan State in Baltimore, North Carolina A&T in Greensboro, Jackson State in Mississippi, and Kentucky State in Frankfort. Additional partners include SRI International, the National CARES Mentoring Network, and the Verizon Foundation. Using the collective impact-style approaches such as planning and implementing a Network Improvement Community (NIC), developing a shared agenda and implementing mutually reinforcing activities, these partners will address two common goals: (1) Broaden the participation of underrepresented minority males in science and engineering through educational experiences that prepare them for careers in STEM fields; and (2) Create a Network Improvement Community focused on STEM achievement in minority males. Program elements include high-quality instruction in STEM content, mentoring, and professional development. The project will expand to include eight additional partners (six HBCUs and two Hispanic-Serving Institutions) and schools and districts in communities local to their campuses. The INCLUDES pilot will help scale innovations that target impacting minorities in STEM.
The project will develop STEM learning pathways for middle school minority males by harnessing the collective impact of 12 university partners, local K-12 schools and districts with which they partner, and surrounding community organizations and businesses with a vested interest in achieving common goals. Products will include a roadmap for addressing the problem through a Network Improvement Community, a website that will contribute to the knowledge base regarding effective strategies for enhancing STEM educational opportunities for minority males, and common metrics, assessments, and shared measurement systems that will be used to measure the collective impact of the Network Improvement Community.
DATE:
-
TEAM MEMBERS:
Jumoke Ladeji-OsiasCindy ZikerGeneva HaertelKamal AliAyanna GillDerrick GilmoreClay Gloster
resourceprojectProfessional Development, Conferences, and Networks
This one-year Collaborative Planning project seeks to bring together an interdisciplinary planning team of informal and formal STEM educators, researchers, scientists, community, and policy experts to identify the elements, activities, and community relationships necessary to cultivate and sustain a thriving regional early childhood (ages 3-6) STEM ecosystem. Based in Southeast San Diego, planning and research will focus on understanding the needs and interests of young Latino dual language learners from low income homes, as well as identify regional assets (e.g., museums, afterschool programs, universities, schools) that could coalesce efforts to systematically increase access to developmentally appropriate informal STEM activities and resources, particularly those focused on engineering and computational thinking. This project has the potential to enhance the infrastructure of early STEM education by providing a model for the planning and development of early childhood focused coalitions around the topic of STEM learning and engagement. In addition, identifying how to bridge STEM learning experiences between home, pre-k learning environments, and formal school addresses a longstanding challenge of sustaining STEM skills as young children transition between environments.
The planning process will use an iterative mixed-methods approach to develop both qualitative and quantitative and data. Specific planning strategies include the use of group facilitation techniques such as World Café, graphic recording, and live polling. Planning outcomes include: 1) a literature review on STEM ecosystems; 2) an Early Childhood STEM Community Asset Map of southeast San Diego; 3) a set of proposed design principles for identifying and creating early childhood STEM ecosystems in low income communities; and 4) a theory of action that could guide future design and research. This project is funded by the Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The NISE Network Professional Impacts Summative Evaluation is a longitudinal examination of individual professionals over the final three years of NISE Net funding. This investigation is based on the NISE Network goals for professionals and explores how involvement with NISE Net impacts an individual professional’s sense of community, learning about nano, and use of nano educational products and practices. This evaluation primarily included professional partners who were: (1) Informal Science Educators (ISE): Professionals from science museums and children’s museums implementing informal
Today institutional and project leaders are faced with two critical dilemmas: (1) building the capacity to respond to the increasing evaluation and accountability demands of funders and stakeholders; and (2) managing the complexities of interconnected, multifaceted, ongoing institutional and cross-institutional work. These challenges require leaders to go beyond traditional approaches to professional development and consider the complex ways that systems of professionals communicate, interact, and evolve. This report draws from three years of research as part of the National Science Foundation
The Complex Adaptive Systems as a Model for Network Evaluations (CASNET) study was a four-year research project investigating evaluation capacity building (ECB) within a network using a complexity theory lens. The study used a case study approach to examine and understand evaluation capacity building within the Nanoscale Informal Science Education Network (NISE Net). NISE Net is a national community of researchers and informal science educators dedicated to fostering public awareness, engagement, and understanding of nanoscale science, engineering, and technology. Instituted in 2005 through NSF funding (DRL-0532536 and 0940143), NISE Net has continuously expanded and is currently comprised of close to 600 science museum and university partners. The intent of the CASNET project was to provide insights on (1) the implications of complexity theory for promoting widespread and systemic use of evaluation within a network, and (2) complex system conditions that foster or impede ECB within a network, i.e., in this case, within the NISE Net.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. The aim of this project is to foster activities in science museums through which public audiences can engage with scientists and engineers in conversations about what synthetic biology is, how research is carried out, and the potential products, outcomes, and implications for society of this work. Researchers and publics will explore personal and societal values and priorities as well as research outcomes so that both groups can learn from each other.
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
The Balboa Park Cultural Partnership, in collaboration with several informal science education and other cultural and business organizations in San Diego, Chicago, and Worcester, MA are implementing a research and development project that investigates a range of possible approaches for stimulating the development of 21st Century creativity skills and innovative processes at the interface between informal STEM learning and methods for creative thinking. The goal of the research is to advance understanding of the potential impacts of creative thinking methods on the public's understanding of and engagement with STEM, with a focus on 21st Century workforce skills of teens and adults. The goal of the project's development activities is to experiment with a variety of "innovation incubator" models in cities around the country. Modeled on business "incubators" or "accelerators" that are designed to foster and accelerate innovation and creativity, these STEM incubators generate collaborations of different professionals and the public around STEM education and other STEM-related topics of local interest that can be explored with the help of creative learning methodologies such as innovative methods to generate creative ideas, ideas for transforming one STEM idea to others, drawing on visual and graphical ideas, improvisation, narrative writing, and the process of using innovative visual displays of information for creating visual roadmaps. Hosting the project's incubators are the Balboa Park Cultural Partnership (San Diego), the Museum of Science and Industry (Chicago) and the EcoTarium (Worcester, MA). National partners are the Association of Science-Technology Centers, the American Association for the Advancement of Science, and the Americans for the Arts. Activities will include: the formation and collaborative processes of three incubator sites, a research study, the development of a creative thinking curriculum infused into science education, professional development based on the curriculum, public engagement events and exhibits, a project website and tools for social networking, and project evaluation. A national advisory council includes professionals in education, science, creativity, and business.
Portal to the Public: Expanding the National Network (PoP: ENN) is implementing around the county the successful NSF-funded Portal to the Public model in which researchers are trained to communicate and interact with the general public at informal science education (ISE) institutions about the research that they are conducting. The project, which follows on a thorough evaluation of the model at eight sites and current implementation at an additional fifteen sites, will incorporate twenty new ISE sites into the growing network, provide training and mentorship to ISE professionals on the use and adaptation of the PoP implementation manual and toolkits, and develop an enhanced network website that will serve as a communication and innovation hub. The work is responsive to the needs and activities of ISE organizations which continue to expand their missions beyond presenting to the public established science, technology, engineering and math (STEM) and are working to become places where visitors can also experience the process and promise of current research via face-to-face interactions with researchers. The project is expanding both the kind and number of institutions involved around the country and is facilitating their capacity to develop a knowledge base, share experiences and best practices.
The aim of this project is to create conversations in science museums among scientists, engineers, and public audiences about an emerging research field, synthetic biology. Synthetic biology applies science and engineering to create new biological systems, and re-design existing biological systems, for useful purposes. This is an important new area of research and development that raises societal questions about potential benefits, costs, and risks. Conversations between researchers and public audiences will focus not only on what synthetic biology is and how research in the field is carried out, but also on the potential products, outcomes, and implications for society of this work. Researchers and publics will explore personal and societal values and priorities as well as desired research outcomes so that both groups can learn from each other. Public participants will benefit from knowing about this field of research, and researchers will benefit from hearing public perspectives directly from the public participants. This project will be led by the Museum of Science with partners at the American Association for the Advancement of Science, the Synthetic Biology Engineering Research Center, the Science Museum of Minnesota, the Ithaca Sciencenter, and several other universities and science museums. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project is aimed at pushing beyond traditional modes of communicating with public audiences rooted in "public understanding of science" modalities into the mechanisms and perspectives associated with "public engagement with science" (PES). The project will support informal educational institutions as facilitators of such PES activities through which mutual learning takes place among research experts and various publics. Formative evaluation will support the development of evaluation tools that practitioners can use themselves to measure impacts of public engagement activities on both scientist and public participants. Summative evaluation will measure the impacts of the project on informal science education practitioners and researchers participating in the development of the project. In the first year of the project, two kinds of engagement activities will be tested at eight pilot sites across the U.S. The first kind will be the focus of "showcase" events, in which researchers demonstrate and talk with museum visitors about the basics of synthetic biology and their research work. The second kind will be the focus of "forum" events in which the multi-directional conversations focus on societal implications and participants' priorities for maximizing the benefits of this new field while minimizing the risks. The work of the first year will inform development of a kit of public engagement materials that will support widespread public engagement with synthetic biology in the second year at up to 200 sites across the U.S. Successful practices and infrastructure developed by the Nanoscale Informal Science Education Network to support NanoDays events will be use for this broad dissemination of public engagement in synthetic biology in year 2. When the project is complete a set of tools and guides will be provided online for developing, implementing, and evaluating engagement events that bring scientists and publics together, specifically about synthetic biology, but adaptable to other emerging research topics. The informal science education field will have a better understanding of how to get scientists, engineers, and publics to engage together in discussions about the societal implications of emerging technologies, and how to evaluate the quality of that engagement for both the researchers and the publics involved. The project will also provide a sense of informed public views on societal issues related to synthetic biology that emerge through a variety of public engagement activities that take place in science museums.