As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This conference proposal represents the first phase of a larger three-phase participatory research project that will use communities of interest as a vehicle for solving problems of common concern about designing youth-based STEM programs. It will set the stage for research over the next 10-25 years about the long-term impact of a variety of youth programs on STEM learning and career aspirations. Through a virtual format, the Association of Science-Technology Centers will bring together two representatives from ten long-standing youth programs, experts in the field of out-of-school time youth programming, and researchers to collaboratively develop a program profile template for measuring the impact of youth programming. The program profile template will help identify specific characteristics that will capture the influence of youth programs on their participation in out-of-school STEM activities.
The program profile template will be the main outcome from the conference. It will serve as the foundation for designing long-term impact studies that support the needs of program staff interested in improving youth programming in informal environments. It will also allow program staff and researchers to document and share intellectual capital, compare goals and features across programs, and support network efforts among informal agencies worldwide. The program profile template will be shared online through informalscience.org, the Association of Science-Technology Centers' communities of practice networks, and through other out-of-school-time national organizations.
Mathematics is a notoriously disliked subject; there is so little stigma associated with being "bad at math", that educated adults openly describe themselves in this way. There are many reasons for math's unpopularity; chief among them is that school mathematics seldom offers opportunities to engage with the richness of this potentially fascinating subject. As a result, the mathematics education pipeline in the United States is more often a filter than a pump, siphoning students out rather than bringing them along. Children have libraries to help them fall in love with literature: where do they get a chance to fall in love with math? This project presents a unique opportunity to study children engaged with mathematics in an informal setting, the Minnesota State Fair, facilitated by mathematically knowledgeable volunteers. The Math On-a-Stick mathematical playground provides a place for children to engage with mathematics by exploring patters, asking quantitative questions, and investigating shape and space to mathematize their play. The project will observe and videotape this engagement to inform the design of mathematical learning environments in a variety of outside of school time settings, such as after school programs and summer camps, that are accessible to a wider range of the population. This project is co-funded by the EHR Core Research (ECR) and Advancing Informal STEM Learning (AISL) programs. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in three thematic areas: STEM learning and learning environments, broadening participation, and STEM workforce development. The AISL program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The project will investigate three research questions: (1) How does the design of various parts of the exhibit differently support rich mathematical interactions between children and mathematicians? (2) How do children engage different parts of the exhibit? How do differences in engagement relate to (a) exhibit design and (b) prior mathematical experience? (3) How do exhibit volunteers, mathematicians, and caregivers interact to support (or undermine) students' mathematical play? The project will use participant observation and videography to capture visitors' activities through the exhibit, analyzing them as qualitative case studies.
This project was submitted in response to EHR Core Research (ECR) program announcement NSF 15-509. The ECR program of fundamental research in STEM education provides funding in critical research areas that are essential, broad and enduring. EHR seeks proposals that will help synthesize, build and/or expand research foundations in the following focal areas: STEM learning, STEM learning environments, STEM workforce development, and broadening participation in STEM. The ECR program is distinguished by its emphasis on the accumulation of robust evidence to inform efforts to (a) understand, (b) build theory to explain, and (c) suggest interventions (and innovations) to address persistent challenges in STEM interest, education, learning, and participation.
In 2015, average mathematics scores on the National Assessment of Educational Progress (NAEP) declined in fourth and eighth grades, the first declines in mathematics at these grade levels since 1990. Declines in U.S. mathematics performance has important implications for overall STEM education as well as STEM workforce and international competitiveness. Researchers at Rutgers University will conduct an analysis to isolate the cause of the mathematics decline by investigating the dimensionality of the NAEP assessment, state-level outcomes, and demographic trends.
The team will use multilevel item response theory modeling techniques to investigate the declines by examining the factor structures to determine dimensionality across years. Researchers will examine subscores corresponding to each dimension of the factor structure at the state and national levels. In addition, subscores will be examined for trends in individual states and jurisdictions. Potentially, the analyses will allow for examination of factors related to state standards adoptions, demographic shifts, and participation rates.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
Well-designed educational games represent a promising technology for increasing students interest in and learning of STEM topics such as physics. This project will research how to optimally combine and embed dynamic assessment and adaptive learning supports within an engaging game design to build effective educational games. The project will add enhancements to a physics game called Physics Playground. The general goal of this research is to test a valid methodology that can be used in the design of next-generation learning games. The enhancement of Physics Playground will leverage the popularity of video games to capture and sustain student attention and teach physics to a much broader audience than is currently the case in traditional physics classrooms. To be most effective, this new genre of learning games needs to not only be highly engaging as a game but also to provide real-time assessment and feedback to students; support understanding of science content (i.e.,Newtonian physics); be accessible to beginners; accommodate a range of proficiencies and interests; and support equity. The research will have particular relevance to designers developing other science games and simulation by providing information about the kinds of learning supports and feedback to students are most effective in promoting engagement and learning. The project is supported by the Cyberlearning and Future Learning Technologies Program, which funds efforts that will help envision the next generation of learning technologies and advance what we know about how people learn in technology-rich environments. Cyberlearning Exploration (EXP) Projects explore the viability of new kinds of learning technologies by designing and building new kinds of learning technologies and studying their possibilities for fostering learning and challenges to using them effectively.
The project will systematically develop, test, and evaluate ways to integrate engaging, dynamic learning supports in Physics Playground to teach formal conceptual physics competencies. More generally, the project aims to advance the learning sciences, particularly in the fields of adaptivity and assessment in educational technology. Using a design-based research approach spanning three years, the research team will: (1) develop and test the effectiveness of various learning support features included in the game in Year 1; (2) develop and test an adaptive algorithm to manage the progression of difficulty in game levels in Year 2; and (3) test learning supports and adaptive sequencing in a controlled evaluation study. This research will provide evidence of the instructional effectiveness of an educational game designed using principles of instructional, game, and assessment design. It will advance understanding of the contributions of different kinds of learning supports (e.g., visualizations and explanations) and adaptivity to game-based learning and contribute to the design of next-generation learning games that successfully blur the distinction between assessment and learning. The project will generate research findings that can be incorporated into other types of STEM learning games.
DATE:
-
TEAM MEMBERS:
Valerie ShuteRussell AlmondFengfeng Ke
Summary
Girlstart’s mission is to increase girls’ interest and engagement in STEM through innovative, nationally-recognized informal STEM education programs. Girlstart examines in this report how STEM education directed toward elementary school girls influences long-term readiness and participation to math and science learning. This report compares Girlstart After School participants’ academic performance to nonparticipant performance. Specifically, it examines how Girlstart After School influences science STAAR performance and course enrollment in subsequent elementary and early middle
This project, a collaboration of teams at Georgia Institute of Technology, Northwestern University, and the Museum of Design Atlanta and the Museum of Science and Industry in Chicago, will investigate how to foster engagement and broadening participation in computing by audiences in museums and other informal learning environments that can transfer to at-home and in-school engagement (and vice versa). The project seeks to address the national need to make major strides in developing computing literacy as a core 21st century STEM skill. The project will adapt and expand to new venues their current work on their EarSketch system which connects computer programming concepts to music remixing, i.e. the manipulation of musical samples, beats and effects. The initiative involves a four-year process of iteratively designing and developing a tangible programming environment based on the EarSketch learning environment. The team will develop three new applications: TuneTable, a multi-user tabletop exhibit for museums; TunePad, a smaller version for use at home and in schools; and an online connection between the earlier EarSketch program and the two new devices.
The goal is to: a) engage museum learners in collaborative, playful programming experiences that create music; b) direct museum learners to further learning and computational music experiences online with the EarSketch learning environment; c) attract EarSketch learners from local area schools to visit the museum and interact with novice TuneTable users, either as mentors in museum workshops or museum guests; and d) inform the development of a smaller scale, affordable tangible-based experience that could be used at homes or in smaller educational settings, such as classrooms and community centers. In addition to the development of new learning experiences, the project will test the hypothesis that creative, playful, and social engagement in the arts with computer programming across multiple settings (e.g. museums, homes, and classrooms) can encourage: a) deeper learner involvement in computer programming, b) social connections to other learners, c) positive attitudes towards computing, and d) the use and recognition of computational concepts for personal expression in music. The project's knowledge-building efforts include research on four major questions related to the goals and evaluation processes conducted by SageFox on the fidelity of implementation, impact, success of the exhibits, and success of bridging contexts. Methods will draw on the Active Prolonged Engagement approach (unobtrusive observation, interviews, tracking-and-timing, data summaries and team debriefs) as well as Participatory Action Research methods.
This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
DATE:
-
TEAM MEMBERS:
Michael HornBrian MagerkoJason Freeman
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program supports new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This project will meet this goal through rigorous research and the broad implementation of an environmental science literacy professional development and learning program for informal educators and youth engaged in outdoor science programs (OSP). With growing support from the literature and the Next Generation Science Standards (NGSS), much attention has been placed on creating and leveraging interdisciplinary science learning opportunities beyond science classrooms. As such, an estimated 300 residential OSPs currently exist in the United States. Unfortunately, the informal educators often charged with facilitating these deep and impactful science learning experiences often lack robust formal training in evidenced-based, age-appropriate environmental science content knowledge and pedagogy specific for the youth in their programs. This issue is often more pronounced in under-resourced and under-served programs and communities. This project will directly address these pervasive challenges in the field by not only providing much needed science focused professional development and resources to informal educators but also by specifically targeting and training informal leaders and educators serving youth in predominately rural areas, low-income communities, and underrepresented communities.
Approximately 200 OSP leaders at 100 OSPs around the country will participate in a week-long, intensive training in the professional development model at one of five regional residential leadership institutes. OSP leaders will then redeliver the training to the approximately 1,500 OSP educators/field instructors in their home institutions. The OSP educators/field instructors will then use what they learn through the professional development to facilitate the environmental science learning program (i.e., curriculum, field experiences, resources, pedagogy) to over 1 million youth (grades 3-8) enrolled in their residential outdoor science programs. In addition, a rigorous implementation study, efficacy study and evaluation will be conducted. The implementation study will investigate: (a) Which of the professional learning model practices were implemented and (b) What successes and challenges the programs faced implementing the model. The mixed methods efficacy study will explore: (a) if outdoor science programs contribute to the development of science learning activation and environmental literacy? and (b) what are the features of these experiences that are correlated with increases in science learning activation and environmental literacy. Approximately 25-35 youth will be randomly selected from each of 50 randomly selected sites to participate in the efficacy study. The data and findings from the research and evaluation produced by this project will contribute to a relatively sparse knowledge and research base specific to youth efficacy and implementation processes and practices across nearly 1/3 of the estimated 300 existing residential outdoor science programs in the United States.
Chemistry is an important and widely relevant field of science. However, when compared with other STEM content areas, chemistry is under-represented in U.S. science museums and other informal educational environments. This project will build, and build knowledge about, innovative approaches to delivering informal science learning activities in chemistry. The project will not only increase public interest and understanding of chemistry but also increase public perception of chemistry's relevance and increase the public's self-efficacy with respect to chemistry. This project outcomes will include a guide for practitioners along with activity materials that will be packaged into a kit, distributed, and replicated for use by informal science educators, chemists, and chemistry students at 250 sites across the U.S. The project team will reach out to organizations that serve diverse audiences and diverse geographic locations, including organizations in rural and inner-city areas. The kits will provide guidance on engaging girls, people with various abilities, Spanish speakers, and other diverse audiences, and include materials in Spanish. Written guides, training videos, and training slides will be included to support training in science communication in general, as well as chemistry in particular. This project is supported by the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings, as a part of its overall strategy to enhance learning in informal environments.
This project will take an innovative approach to develop informal educational activities and materials about chemistry. Rather than starting with content goals, the project will start with a theoretical framework drawn from research about affecting attitudes about science related to interest, relevance, and self-efficacy. A design-based research approach (DBR) will be used to apply that framework to the development of hands-on educational activities about chemistry, while also testing and modifying the framework itself. (DBR blends empirical educational research with the theory-driven design of learning environments.) Existing or new educational activities that appear to embody key characteristics defined in the framework will be tested with public audiences for their impact on visitors. Researchers and educators will determine how different characteristics of the educational activities defined in the framework affect the outcomes. The activities will be modified and tested iteratively until the investigators achieve close alignment between framework and impacts.. The project team will continue the design-based research approach both to examine groups of activities in which synergies can have impacts beyond single interactions as well as to examine varied ways of training facilitators who can also significantly affect outcomes. In this way, the project will generate knowledge about how kits of hands-on informal learning activities can stimulate attitudes of interest, relevance, and self-efficacy with respect to the neglected field of chemistry. The project teams will broadly disseminate project outcomes within the educational research, science and informal Science, Technology, Engineering and Mathematics (STEM) education communities. While this project will focus on chemistry, the strategies it will develop and test through a design-based research process will provide valuable insight into effective approaches for informal STEM education more broadly.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. The National Association of Math Circles (NAMC) will convene the Math Circle-Mentor and Partnership (MC-MAP) Workshop in late 2016. The proposed MC-MAP workshop will build the field's understanding of the training content and mechanisms that enhance the knowledge and skill development of participants in Math Circles. The workshop will bring mentors from experienced Math Circle leaders together with novice Math Circle leaders to develop the expertise of the notice leaders and their group to develop their expertise in facilitating math circle activities and in organizing related events. The approximately 180 Math Circles currently operating across the nation enlist mathematics professionals to share their passion for mathematics with K-12 students, teachers, and the general public in contexts that emphasize exploration, problem solving and discovery. This initial conference and Math Circle trainings informed by this conference will help build a community of practice around Math Circles through which novice and existing leaders are connected, encouraged and inspired.
The MC-MAP workshop will include structured planning as well as guided observation and structured debriefing of a demonstration Math Circle sessions. The workshop design will be grounded in research related to effective adult learning and to discovery-based mathematics. The workshop will serve as a training prototype that will assist the National Association of Math Circles to identify effective training formats and materials for both experienced and novice Math Circle leaders. Pre- and post- conference surveys of Math Circle leaders will produce data to be used in planning and designing future trainings. The NAMC will share key findings from the workshop evaluation and workshop resources not only with its membership, but also with other mathematics K-12 outreach programs. Workshop materials will address recruiting and serving diverse participants in Math Circles, including girls and women, persons with disabilities, students from varied socioeconomic backgrounds and underrepresented minorities in STEM.
Youth environmental education (EE) programs often serve as gateway experiences in which diverse audiences engage in informal science learning. While there is evidence that these programs can have positive impacts on participants, little empirical research has been conducted to determine what makes one program more successful than another. To be able to conduct such research, this Exploratory Pathways study will (1) develop and statistically validate ways to measure meaningful outcomes for participants across a variety of programs and (2) test observational methods that will enable research that can determine which elements of program delivery most powerfully influence participant engagement and learning outcomes in different contexts. These efforts will include consultations with diverse subject matter experts from the National Park Service, nature centers, and academia; survey research with participants in afterschool and free-choice EE programs; and observations of EE programs designed to fine tune the measurement of program delivery elements and student engagement. Developing valid and reliable outcomes measures and observational protocols will enable a larger investigation that will specifically address the following research question: What program characteristics lead to the best learning outcomes for program participants in different contexts? This research will result in empirically tested guidelines that will enable educators to design and deliver more effective programs for a wide range of audiences in a wide range of contexts. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This effort will refine methods necessary to undertake an unprecedented study (and future AISL Research in Service to Practice proposal) to examine the linkages between pedagogical approaches, participant engagement, and learning outcomes in informal STEM-focused youth EE programs. The larger study will involve systematically observing a large number of programs to assess the use of different approaches and to link those approaches to engagement and learning outcomes through both observation and survey research. In this current study the team will develop and refine crosscutting outcome measures to ensure validity, reliability, and sensitivity by drawing upon the literature and consultation with key stakeholders to develop suites of indicators for subsequent psychometric testing and revision. They will also refine observational techniques for assessing pedagogical approaches through extensive testing of inter-rater reliability. Finally, techniques for measuring participant engagement, incorporating both observational techniques and retrospective participant surveys will be refined. The work will be conducted by researchers at Clemson University and Virginia Tech, in partnership with the U.S. National Park Service, the North American Association for Environmental Education, and the American Association of Nature Center Administrators. This work represents the first step in a longer research process to determine the "best practices" most responsible for achieving outcomes in a wide range of contexts.