While the term 'failure' brings to mind negative associations, there is a current focus on failure as a driver of innovation and development in many professional fields. It is also emerging from prior research that for STEM professionals and educators, failure plays an important role in designing and making to increase learning, persistence and other noncognitive skills such as self-efficacy and independence. By investigating how youth and educators attend to moments of failure, how they interpret what this means, and how they respond, we will be better able to understand the dynamics of each part of the experience. The research team will be working with youth from urban, suburban and rural settings, students from Title I schools or who qualify for free/reduced-price lunches, those from racial and ethnic minority groups, as well as students who are learning English as a second language. These youth are from groups traditionally underrepresented in STEM and in making, and research indicates they are more likely to experience negative outcomes when they experience failure.
The intellectual merit of this project centers on establishing a baseline understanding of how failure in making is triggered and experienced by youth, what role educators play in the process, and what can be done to increase persistence and learning, rather than failure being an end-state. The research team will investigate these issues through the use of qualitative and quantitative research methods. In particular, the team will design and evaluate the effectiveness of interventions on increasing the abilities of youth and educators in noticing and responding to failures and increasing positive (e.g., resilience) outcomes. Research sites are selected because they will allow collection of data on youth from a wide range of backgrounds. The research team will also work to test and revise their hypothesized model of the influence of factors on persistence through failures in making. This project is a part of NSF's Maker Dear Colleague Letter (DCL) portfolio (NSF 15-086), a collaborative investment of Directorates for Computer & Information Science & Engineering (CISE), Education and Human Resources (EHR) and Engineering (ENG).
This Research in Service to Practice project, a collaboration of Pepperdine University and the New York Hall of Science, will establish a network of STEM-related Media Making Clubs comprised of after-school students aged 12 - 19 and teachers in the U.S. and in three other countries: Kenya, Namibia and Finland. The media produced by the students may include a range of formats such as videos, short subject films, games, computer programs and specialized applications like interactive books. The content of the media produced by the students will focus on the illustration and teaching of STEM topics, where the shared media is intended to help other students become enthused about and learn the science. This proposal builds on the principal investigator's previous work on localized media clubs by now creating an international network in which after-school students and teachers will collaborate at a distance with other clubs. The central research questions for the project pertain to three themes at the intersection of learning, culture and collaboration: the impact of participatory teaching, virtual networks, and intercultural, global competence. The research will combine qualitative, cross-cultural and big data methods. Critical to the innovation of the project, the research team will also develop a network assessment tool, adapting epistemic network analysis methods to the needs of this initiative. This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
DATE:
-
TEAM MEMBERS:
Eric HamiltonKatherine McMillanPriya Mohabir
This project by California State University San Marcos and their collaborators will expand and continue to innovate on a pilot Mobile Making program with the goal of developing a sustainable, regional model for serving underserved, middle-school aged youth in twelve after-school programs in the San Diego region. Evaluation of the current Mobile Making program has documented positive impacts on participants' interest, self-efficacy, and perception of the relevance of Making/STEM in everyday life, and led to a model for engaging underserved youth in Making. The work will focus on implementing the program model sustainably at greater capacity by increasing the number of undergraduate activity leaders, after-school sites, and level of community engagement. The expanded Mobile Making program is expected to engage ~1800 middle school youth at 12 local school sites, with activities facilitated by ~1020 undergraduate CSU-SM STEM majors. The sites are in ethnically diverse and economically disadvantaged neighborhoods, with as many as 90% of students at some sites qualifying for free or reduced price lunch. The undergraduate facilitators are drawn from CSU-SM's diverse student body, which includes 44% underrepresented minorities. Outcomes are expected to include increases in the youth participants' interest, self-efficacy, and perception of the relevance of Making/STEM in everyday life. Positive impacts on the undergraduate facilitators will include broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. The program is designed to achieve sustainability through innovative means such as involving undergraduate facilitators via Community Service Learning (rather than paid positions), and increased community engagement via development and support of a community of practice including local after-school providers, teachers, Makers, and University members. Evaluation of the program outcomes and lessons learned are expected to result in a comprehensive model for a sustainable, university-based after-school Making program with regional impact in underserved communities. Dissemination to other regions will be leveraged via CSU-SM's membership in the California State University (CSU) system, yielding a potential statewide impact. The support of the CSU Chancellor's Office and input from a CSU implementation group will ensure the applicability of the model to other regional university settings, identify common structural barriers and solutions, and increase the probability of secondary implementations. This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Researchers at Georgia Tech and staff at the Center for Puppetry Arts in Atlanta will address the following problem: The growth of computing calls for increased teaching of basic hardware skills, but engaging students beyond those who are already interested in STEM fields is difficult. Emphasizing cultural context through personal expression has shown to attract new audiences such as women and minorities; however, balancing accessibility and educational effectiveness remains problematic. Current educational tools often use "black boxing" to simplify access, to protect the underlying functionality, and/or to stay commercially competitive. Innovations are needed where the basic skills of design and making, along with new educational models, can support students' exploration of everyday electronics and critical thinking. The team will test an after-school model that could be applied in cultural institutions and other venues that would provide students in 4 - 8 grades with multiple computer/electronics-driven puppet design and building workshops. These workshops will provide opportunities for basic hardware and electronics prototyping, personal cultural expression, and performance.
The exploratory research and development method involves three steps: the students develop a story-line (narrative) that will incorporate puppets; they learn and apply basic electronics and mechanical crafts to design, build and test their puppets; and they perform their shows to other participants and their families. Over the course of the project, there will be five workshops: two pilot tests (to test feasibility and design), one mid-project evaluation workshop (to evaluate and re-iterate), and two final workshops conducted by educators, where the research team only observes (to test for transferability). Data collection throughout this development cycle will include video and sound recording, still photography, field notes, pre- and post-questionnaires, and the production of puppet "artifacts" during the workshops. In addition to the data collected in the workshops, the project will produce design documents and educational guidelines, as well as other media (such as instructional video clips), which will be disseminated to educators and researchers.
The project has been successfully concluded. It included additional pilot workshops with puppetry experts and educators in addition to the scheduled development and final evaluation workshops. Overall, the project has led to publications in both educational as well as performance/ game- and interface-related events.
The project also published material on the puppet designs and their possible adaptation to educational settings on the project web site, which also include video documentation. More information is available at: http://dwig.lmc.gatech.edu/projects/prototypingpuppets/
As a part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings. This Broad Implementation project would scale up the CryptoClub Project, an afterschool and online program designed to engage middle school youth in mathematics and cryptography. The project builds on previous successful work and evaluation that is ready for scale up using a train-the-trainer model implemented through a partnership with the National Girls Collaborative. The project will train 160 new CryptoClub leaders who will then train 800 new leaders at 20 hub sites reaching 9600 students. In addition, professional development modules and webinars will continue to refresh leader skills. Other project components include an online multiplayer cryptography game, weekly challenges through social media, and digital cryptology badges for students.
The research uses a think-aloud method with students as they actually attempt to solve the cryptology problems using mathematical thinking. Three think-aloud studies will be performed during the Project. The research team will code transcripts of the interviews for evidence of the mathematical thinking intended to be addressed by each activity, as well as capturing unexpected kinds of thinking. Tasks will also be rated according to the type of knowledge elicited. A written report will include statistical analyses of the think-aloud and interview responses, interpreted in light of the overall CryptoClub goals. The findings will contribute to both future research efforts and practice. The evaluation by EDC uses a quasi-experimental design, which assesses project outcomes for trainers, leaders, students, and Internet users. EDC will also investigate the fidelity to the CryptoClub model as it is scaled up. These studies have strong potential for informing numerous other projects that are at a stage where scale up is under consideration.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program supports new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This project will meet this goal through rigorous research and the broad implementation of an environmental science literacy professional development and learning program for informal educators and youth engaged in outdoor science programs (OSP). With growing support from the literature and the Next Generation Science Standards (NGSS), much attention has been placed on creating and leveraging interdisciplinary science learning opportunities beyond science classrooms. As such, an estimated 300 residential OSPs currently exist in the United States. Unfortunately, the informal educators often charged with facilitating these deep and impactful science learning experiences often lack robust formal training in evidenced-based, age-appropriate environmental science content knowledge and pedagogy specific for the youth in their programs. This issue is often more pronounced in under-resourced and under-served programs and communities. This project will directly address these pervasive challenges in the field by not only providing much needed science focused professional development and resources to informal educators but also by specifically targeting and training informal leaders and educators serving youth in predominately rural areas, low-income communities, and underrepresented communities.
Approximately 200 OSP leaders at 100 OSPs around the country will participate in a week-long, intensive training in the professional development model at one of five regional residential leadership institutes. OSP leaders will then redeliver the training to the approximately 1,500 OSP educators/field instructors in their home institutions. The OSP educators/field instructors will then use what they learn through the professional development to facilitate the environmental science learning program (i.e., curriculum, field experiences, resources, pedagogy) to over 1 million youth (grades 3-8) enrolled in their residential outdoor science programs. In addition, a rigorous implementation study, efficacy study and evaluation will be conducted. The implementation study will investigate: (a) Which of the professional learning model practices were implemented and (b) What successes and challenges the programs faced implementing the model. The mixed methods efficacy study will explore: (a) if outdoor science programs contribute to the development of science learning activation and environmental literacy? and (b) what are the features of these experiences that are correlated with increases in science learning activation and environmental literacy. Approximately 25-35 youth will be randomly selected from each of 50 randomly selected sites to participate in the efficacy study. The data and findings from the research and evaluation produced by this project will contribute to a relatively sparse knowledge and research base specific to youth efficacy and implementation processes and practices across nearly 1/3 of the estimated 300 existing residential outdoor science programs in the United States.
Chemistry is an important and widely relevant field of science. However, when compared with other STEM content areas, chemistry is under-represented in U.S. science museums and other informal educational environments. This project will build, and build knowledge about, innovative approaches to delivering informal science learning activities in chemistry. The project will not only increase public interest and understanding of chemistry but also increase public perception of chemistry's relevance and increase the public's self-efficacy with respect to chemistry. This project outcomes will include a guide for practitioners along with activity materials that will be packaged into a kit, distributed, and replicated for use by informal science educators, chemists, and chemistry students at 250 sites across the U.S. The project team will reach out to organizations that serve diverse audiences and diverse geographic locations, including organizations in rural and inner-city areas. The kits will provide guidance on engaging girls, people with various abilities, Spanish speakers, and other diverse audiences, and include materials in Spanish. Written guides, training videos, and training slides will be included to support training in science communication in general, as well as chemistry in particular. This project is supported by the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings, as a part of its overall strategy to enhance learning in informal environments.
This project will take an innovative approach to develop informal educational activities and materials about chemistry. Rather than starting with content goals, the project will start with a theoretical framework drawn from research about affecting attitudes about science related to interest, relevance, and self-efficacy. A design-based research approach (DBR) will be used to apply that framework to the development of hands-on educational activities about chemistry, while also testing and modifying the framework itself. (DBR blends empirical educational research with the theory-driven design of learning environments.) Existing or new educational activities that appear to embody key characteristics defined in the framework will be tested with public audiences for their impact on visitors. Researchers and educators will determine how different characteristics of the educational activities defined in the framework affect the outcomes. The activities will be modified and tested iteratively until the investigators achieve close alignment between framework and impacts.. The project team will continue the design-based research approach both to examine groups of activities in which synergies can have impacts beyond single interactions as well as to examine varied ways of training facilitators who can also significantly affect outcomes. In this way, the project will generate knowledge about how kits of hands-on informal learning activities can stimulate attitudes of interest, relevance, and self-efficacy with respect to the neglected field of chemistry. The project teams will broadly disseminate project outcomes within the educational research, science and informal Science, Technology, Engineering and Mathematics (STEM) education communities. While this project will focus on chemistry, the strategies it will develop and test through a design-based research process will provide valuable insight into effective approaches for informal STEM education more broadly.
Youth environmental education (EE) programs often serve as gateway experiences in which diverse audiences engage in informal science learning. While there is evidence that these programs can have positive impacts on participants, little empirical research has been conducted to determine what makes one program more successful than another. To be able to conduct such research, this Exploratory Pathways study will (1) develop and statistically validate ways to measure meaningful outcomes for participants across a variety of programs and (2) test observational methods that will enable research that can determine which elements of program delivery most powerfully influence participant engagement and learning outcomes in different contexts. These efforts will include consultations with diverse subject matter experts from the National Park Service, nature centers, and academia; survey research with participants in afterschool and free-choice EE programs; and observations of EE programs designed to fine tune the measurement of program delivery elements and student engagement. Developing valid and reliable outcomes measures and observational protocols will enable a larger investigation that will specifically address the following research question: What program characteristics lead to the best learning outcomes for program participants in different contexts? This research will result in empirically tested guidelines that will enable educators to design and deliver more effective programs for a wide range of audiences in a wide range of contexts. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This effort will refine methods necessary to undertake an unprecedented study (and future AISL Research in Service to Practice proposal) to examine the linkages between pedagogical approaches, participant engagement, and learning outcomes in informal STEM-focused youth EE programs. The larger study will involve systematically observing a large number of programs to assess the use of different approaches and to link those approaches to engagement and learning outcomes through both observation and survey research. In this current study the team will develop and refine crosscutting outcome measures to ensure validity, reliability, and sensitivity by drawing upon the literature and consultation with key stakeholders to develop suites of indicators for subsequent psychometric testing and revision. They will also refine observational techniques for assessing pedagogical approaches through extensive testing of inter-rater reliability. Finally, techniques for measuring participant engagement, incorporating both observational techniques and retrospective participant surveys will be refined. The work will be conducted by researchers at Clemson University and Virginia Tech, in partnership with the U.S. National Park Service, the North American Association for Environmental Education, and the American Association of Nature Center Administrators. This work represents the first step in a longer research process to determine the "best practices" most responsible for achieving outcomes in a wide range of contexts.
The Wayne State University Math Corps is a mathematics enrichment and mentoring program that operates during summers and on Saturdays. The curriculum and the teach pedagogies in this informal learning program have documented success of supporting youths' mathematics learning as well as raising achievement levels in school. Through rigorous research and evaluation, this project seeks to analyze and understand the nature, extent, and reasons for Math Corps' success with youth learning in Detroit as well as the processes of program replication in three sites: Cleveland, OH; Utica, NY; and Philadelphia, PA. As such, this project will deepen understandings of program replication and of addressing the needs of youth in economically-challenged communities in order to promote mathematics learning.
The project's research studies will assess the multiple factors that make Math Corps successful with youth in Detroit and document the implementation of the program to the three replication sites. Research methods include discourse analyses, surveys, interviews, and pre/post-tests. The project will also conduct a retrospective evaluation of Math Corps based on quantitative datasets regarding both near-term and long-term youth outcomes.
This projects is funded by the Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understandings of, the design and development of STEM learning in informal environments.
DATE:
-
TEAM MEMBERS:
Steve KahnStephen ChrisomalisTodd KubicaCarol Philips-BeyFrancisca Richter
In this chapter we present the ways in which institutional cultural differences impact the development and implementation of learning activities in informal settings. Five university-based centers for the study of chemistry worked with informal learning professionals to re-envision educational and public outreach activities about science. The projects were part of a broader effort to catalyze new thinking and innovation in informal education and chemistry centers. The set of projects illustrates the broad possibilities for informal learning settings, with projects targeting diverse audiences
Situated within the Advancing Informal STEM Learning program, this Research in Service to Practice award seeks to design, implement, and evaluate an intervention aimed at fostering a culture of productive failure practices. The project responds to a broad concern in educational research and practice: Experiences of failure are frequently so negative that students shut down, lose agency, and develop low self-efficacy and learned helplessness. Surrendering too quickly to obstacles is particularly unfortunate, given evidence that initially "getting it wrong" ultimately breeds deep and sustained learning. In order to learn how students can make the most of productive failure, the proposed project will study how a community of practice that includes middle school youth and their mentors attempts to change its handling of learning obstacles. Building on prior research documenting storytelling practices in an afterschool program, the team now aims to embolden young students' productive practices of failure storytelling in computer science, a field in which experts practice candid, pervasive, and collaborative discourse around errors ("bugs"). Pulling together the domains of narrative analysis, meta-cognitive reflection, and control theories of motivation, within the context of authentic computer-science debugging activity, this study develops a theoretical framework that views productive responses to failure as a discipline-specific process of reflecting as a community on how to locate obstacles, how to construct causal theories about why those obstacles emerged, and how to plan productive responses. A design-based research approach will investigate three questions: (1) What is the impact of the interventions on students and instructors' actions and discourse when they are debugging errors in computer code? (2) What is the impact of the interventions on students and instructors' reflections back on their prior debugging experiences and on failure in general? and (3) What is the impact of the instructor-development efforts on the instructors' capacity to foster students' productive attitudes toward failure? The study focus will be 15 summer and weekend coding workshops with 5th-8th grade students from populations typically under-represented in STEM. The interventions are (a) setting new norms and practices for debugging, (b) instructor education, and (c) coding software that provides students with feedback on their productive struggle. Data sources include video and audio recordings of the learning environment, artifacts produced during the activities, and semi-structured interviews. Measures will capture variations in debugging activities, reflections on debugging, students' ideas about grit and growth mindset, and instructors' struggles and successes with the new curriculum. The empirical results will consist of mixed-methods, micro-longitudinal accounts of how a community of practice works to reform its orientation to failure. The products of this work include empirical knowledge, theory, and curriculum about how learning communities help students develop robust and efficient responses to failure. These will be disseminated through journals, open-source software, and workshops/conferences for researchers and practitioners working with youth afterschool programs. The products may be useful for exploring practices in the classroom. This project is being conducted by the 9 Dots Community Learning Center, UCLA and UC Berkeley.
DATE:
-
TEAM MEMBERS:
Melissa ChenDor AbrahamsonNoel EnyedyFrancis SteenDavid DeLiema
Currently, many young people - especially girls and youth of color - lose confidence and interest in science, technology, engineering and math (STEM) pathways due to a perceived disconnect between their own identity and STEM fields. To address this challenge, Twin Cities PBS (TPT) is implementing SciGirls CONNECT2. This three-year Research in Service to Practice award examines how gender equitable and culturally responsive teaching strategies influence middle school girls' confidence, interest and motivation around STEM studies, and their choices around STEM careers. A set of research-based strategies, called the SciGirls Seven, are currently employed in SciGirls, an NSF-funded informal STEM educational outreach program serving 125+ educational partner organizations nationwide. The goal is to update and enrich the SciGirls Seven, providing educators with a critical, current, and more effective resource to motivate girls in STEM studies and careers. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Florida State University will conduct a formal research study investigating the hypothesis that STEM programs that use gender equitable and culturally responsive strategies contribute to girls' positive STEM identity development, including their sense of self-efficacy, persistence and aspirations around future STEM careers. This research will include a literature review and a study of girls' STEM identity creation. The mixed methods study will include quantitative and qualitative data collection and analysis measuring changes in students' STEM identity and teachers' confidence in STEM teaching. The quantitative data will come from the student, parent and teacher pre/post surveys. The qualitative research will be conducted via case studies at four sites and the qualitative data will include observations, focus groups and interviews. Girls at all partner sites will create videos that will allow the research team to gather additional insight. The independent firm Knight Williams, Inc. will conduct the project's external evaluation.
The project will work with a subset of 16 current SciGirls partners. These geographically diverse partners will reach youth in all-girls and co-ed informal STEM education programs in a variety of settings. More than half serve Hispanic or other minority populations. The updated strategies will be disseminated to the 2,500 educators within the SciGirls partner network and the 18,800 STEM education organizations of the National Girls Collaborative Project (NGCP) network. Dissemination of the strategies and literature review will focus on the informal STEM education field through publications and presentations, posts at PBS LearningMedia, a free online space reaching 1.5 million teachers and educators.