Out-of-school time (OST) youth programs are inherently difficult to assess. They are often very dynamic: Many youth interact with one another and with staff members in various physical environments. Despite the challenge, measuring quality is critical to help program directors and policy makers identify where to improve and how to support those improvements.
DATE:
TEAM MEMBERS:
Allison TracyLinda CharmaramanIneke CederAmanda RicherWendy Surr
STEM learning is a process that unfolds through dynamic interactions over time and across settings. Formal education in schools is not the only—or necessarily the most significant—context for STEM learning.
A skilled workforce is critical in high-quality out-of-school time (OST) programs (Smith, Devaney, Akiva & Sugar, 2009). However, the workshops commonly used to train OST staff are not adequately preparing practitioners to deliver quality programs that can benefit youth(Durlak & Weissberg, 2007; Smith et al, 2009).
DATE:
TEAM MEMBERS:
Femi VanceEmily SalvaterraJocelyn Atkins MichelsenCorey Newhouse
According to the Harvard Family Research Project (2010), schools need collaborative partners to help children and youth thrive. For over a decade, afterschool programs have been positioning themselves as viable partners. After all, afterschool programs challenge students’ thinking, teach collaboration, and help children and youth find their passion.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
Summary
Girlstart’s mission is to increase girls’ interest and engagement in STEM through innovative, nationally-recognized informal STEM education programs. Girlstart examines in this report how STEM education directed toward elementary school girls influences long-term readiness and participation to math and science learning. This report compares Girlstart After School participants’ academic performance to nonparticipant performance. Specifically, it examines how Girlstart After School influences science STAAR performance and course enrollment in subsequent elementary and early middle
This Research in Service to Practice project, a collaboration of Pepperdine University and the New York Hall of Science, will establish a network of STEM-related Media Making Clubs comprised of after-school students aged 12 - 19 and teachers in the U.S. and in three other countries: Kenya, Namibia and Finland. The media produced by the students may include a range of formats such as videos, short subject films, games, computer programs and specialized applications like interactive books. The content of the media produced by the students will focus on the illustration and teaching of STEM topics, where the shared media is intended to help other students become enthused about and learn the science. This proposal builds on the principal investigator's previous work on localized media clubs by now creating an international network in which after-school students and teachers will collaborate at a distance with other clubs. The central research questions for the project pertain to three themes at the intersection of learning, culture and collaboration: the impact of participatory teaching, virtual networks, and intercultural, global competence. The research will combine qualitative, cross-cultural and big data methods. Critical to the innovation of the project, the research team will also develop a network assessment tool, adapting epistemic network analysis methods to the needs of this initiative. This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
DATE:
-
TEAM MEMBERS:
Eric HamiltonKatherine McMillanPriya Mohabir
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This Research in Service to Practice project will examine how a wide range of pre-college out-of-school-time activities facilitate or hinder females' participation in STEM fields in terms of interest, identity, and career choices. The study will address the ongoing problem that, despite females' persistence to degree once declaring a major in college, initially fewer females than males choose a STEM career path. To uncover what these factors might be, this study will look at the extent to which college freshmen's pre-college involvement in informal activities (e.g., science clubs, internships, shadowing of STEM professionals, museum-going, engineering competitions, citizen science pursuits, summer camps, and hobbies) is associated with their career aspirations and avocational STEM interests and pursuits. While deep-seated factors, originating in culture and gender socialization, sometimes lower females' interest in STEM throughout schooling, this study will examine the degree to which out-of-school-time involvement ameliorates the subtle messages females encounter about women and science that can interfere with their aspiration to a STEM careers.
The Social Cognitive Career Theory will serve as the theoretical framework to connect the development of interest in STEM with students' later career choices. An epidemiological approach will be used to test a wide range of hypotheses garnered from a review of relevant literature, face-to-face or telephone interviews with stakeholders, and retrospective online surveys of students. These hypotheses, as well as questions about the students' demographic background and in-school experiences, will be incorporated into the main empirical instrument, which will be a comprehensive paper-and-pencil survey to be administered in classes, such as English Composition, that are compulsory for both students with STEM interests and those without by 6500 students entering 40 large and small institutions of higher learning. Data analysis will proceed from descriptive statistics, such as contingency tables and correlation matrices, to multiple regression and hierarchical modeling that will link out-of-school-time experiences to STEM interest, identity, and career aspirations. Factor analysis will be used to combine individual out-of-school activities into indices. Propensity score weighting will be used to estimate causal effects in cases where out-of-school-time activities may be confounded with other factors. The expected products will be scholarly publications and presentations. Results will be disseminated to out-of-school-time providers and stakeholders, educators, and educational researchers through appropriate-level journals and national meetings and conferences. In addition, the Public Affairs and Information Office of the Harvard-Smithsonian Center for Astrophysics will assist with communicating results through mainstream media. Press releases will be available through academic outlets and Op-Ed pieces for newspapers. The expected outcome will be research-based evidence about which types of out-of-school STEM experiences may be effective in increasing young females' STEM interests. This information will be crucial to educators, service providers, as well as policy makers who work toward broadening the participation of females in STEM.
DATE:
-
TEAM MEMBERS:
Roy GouldPhilip SadlerGerhard Sonnert
Currently, many young people - especially girls and youth of color - lose confidence and interest in science, technology, engineering and math (STEM) pathways due to a perceived disconnect between their own identity and STEM fields. To address this challenge, Twin Cities PBS (TPT) is implementing SciGirls CONNECT2. This three-year Research in Service to Practice award examines how gender equitable and culturally responsive teaching strategies influence middle school girls' confidence, interest and motivation around STEM studies, and their choices around STEM careers. A set of research-based strategies, called the SciGirls Seven, are currently employed in SciGirls, an NSF-funded informal STEM educational outreach program serving 125+ educational partner organizations nationwide. The goal is to update and enrich the SciGirls Seven, providing educators with a critical, current, and more effective resource to motivate girls in STEM studies and careers. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Florida State University will conduct a formal research study investigating the hypothesis that STEM programs that use gender equitable and culturally responsive strategies contribute to girls' positive STEM identity development, including their sense of self-efficacy, persistence and aspirations around future STEM careers. This research will include a literature review and a study of girls' STEM identity creation. The mixed methods study will include quantitative and qualitative data collection and analysis measuring changes in students' STEM identity and teachers' confidence in STEM teaching. The quantitative data will come from the student, parent and teacher pre/post surveys. The qualitative research will be conducted via case studies at four sites and the qualitative data will include observations, focus groups and interviews. Girls at all partner sites will create videos that will allow the research team to gather additional insight. The independent firm Knight Williams, Inc. will conduct the project's external evaluation.
The project will work with a subset of 16 current SciGirls partners. These geographically diverse partners will reach youth in all-girls and co-ed informal STEM education programs in a variety of settings. More than half serve Hispanic or other minority populations. The updated strategies will be disseminated to the 2,500 educators within the SciGirls partner network and the 18,800 STEM education organizations of the National Girls Collaborative Project (NGCP) network. Dissemination of the strategies and literature review will focus on the informal STEM education field through publications and presentations, posts at PBS LearningMedia, a free online space reaching 1.5 million teachers and educators.
As part of an overall strategy to enhance learning within maker contexts in formal and informal environments, the Innovative Technology Experiences for Students and Teachers (ITEST) and Advancing Informal STEM Learning (AISL) programs partnered to support innovative models for making in a variety of settings through the Enabling the Future of Making to Catalyze New Approaches in STEM Learning and Innovation Dear Colleague Letter. This Early Concept Grant for Exploratory Research (EAGER) will test an innovative approach to bringing making from primarily informal out-of-school contexts into formal science classrooms. While the literature base to support the positive outcomes and impacts of design-based making in informal settings at the K-12 level is emerging, to date, minimal studies have investigated the impacts of making design principles within formal contexts. If successful, this project would not only add to this gap in the literature base but would also present a novel model for bridging the successful engineering design practices of making and tinkering primarily found in informal science education into formal science education classrooms. The model would also demonstrate an innovative, highly interactive way to engage high school students and their teachers in engineering based design principles with immediate real-world applications, as the scientific instruments developed in this project could be integrated directly into science classrooms at relatively minimal costs.
Through a multi-phased design and implementation model, high school students and their teachers will engage deeply in making design principles through the design and development of their own scientific instruments using Arduino-compatible hardware and software. The first phase of the project will reflect a more traditional making experience with up to twenty high school students and their teachers participating in an after-school design making club, in this case, focused on the development and testing of scientific instrument prototypes. During the second phase of the project, the first effort to transpose the after school making experience to a more formalized experience will be tested with up to eight students selected to participate in two week summer research internships focused on scientific instrument design and development through making at Northwestern University. A two-day summer teacher workshop will also be held for high school teachers participating in the subsequent pilot study. The collective insights gleaned from the after school program, student internships, and teacher workshop will culminate to inform the full implementation of the formal classroom pilot study. The third and final phase will coalesce months of iterative, formative research, design and development, resulting in a comprehensive pilot investigation in up to seven high school physics classrooms.
Using a multi-phased, mixed methods exploratory design-based research approach, this 18-month EAGER will explore several salient research questions: (a) How and to what extent does the design & making of scientific instrumentation serve as useful tasks for learning important science and engineering knowledge, practices, and epistemologies? (b) How engaging is this making activity to learners of diverse abilities and prior interests? What can be generalized to other types of making activities? (c) How accessible is the Arduino hardware and coding environment to learners? What combination of hardware and software materials and tools best support accessibility and learning in this type of digital making activity? and (d) What types of scaffolding (for students and teachers) are required to support the effective use of maker materials and activities in a classroom setting? Structured interviews, artifacts, video recordings from visor cameras, student design logs, logfiles, and ethnographic field notes will be employed to garner data and address the research questions. Given the early stage of the proposed research, the dissemination of the findings will be limited to a few select journals, teacher forums and workshops, and professional conferences.
This EAGER is well-poised to directly impact up to 125 high school physics students (average= 25 students/class), approximately 7 high school physics teachers, 6-8 high school summer interns, nearly 20 high school students participating in the after-school design making club, and indirectly many more. The results of this EAGER could provide the basis and evidence needed to support a more robust, expanded future investigation to further substantiate the findings and build the case for similar efforts to bring making into formal science education contexts.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. CHISPA is a national network of science museums and afterschool programs affiliated with ASPIRA and National Council of La Raza (NCLR), working together to build stronger communities and increase the engagement of Hispanic children and their families with science and local science resources. The project period is October 2013 through September 2017.